Vector search for fashion products: how semantic similarity improves discovery

Baymard Institute reports that 31 per cent of all product searches in online shops return no result even though the requested product exists in the catalogue. Classical keyword search based on BM25 fails at synonyms, intent variants and long-tail queries. Vector search powered by embeddings addresses this gap by converting product images, titles and attributes into mathematical representations that enable search systems to match semantic similarity rather than exact keywords. Search remains the primary mode of online product discovery, with 69 per cent of customers stating they go directly to a retailer's search bar when shopping online. However, 80 per cent are dissatisfied with the search experience and leave the site as a result, with 41 per cent citing irrelevant results as a main barrier to shopping.

What are embeddings and how do they work in fashion search?

Embeddings convert text, images, or other data into numerical vectors, allowing you to measure similarity between products. Visual search requires a catalogue embedding pipeline where all product images are embedded into a high-dimensional vector representation at index time, then search queries are embedded at query time, and approximate nearest-neighbour search retrieves the most visually similar catalogue items.

For text-based search, models like BERT, Sentence-BERT or Universal Sentence Encoder are strong candidates. These models capture semantic relationships, which is critical for handling synonyms such as "cellphone" versus "mobile phone" or varied product attributes. For image-based search, models like CLIP or ResNet generate embeddings that link visual features to textual queries. CLIP-family models produce image embeddings that align well with text semantics, useful for cross-modal retrieval where a shopper submits a text-described image or the retrieval needs to incorporate product metadata.

Which embedding strategy delivers the best results for fashion catalogues?

Elastic benchmarked 5,000 real apparel and footwear products in English and German across two Jina embedding models in July 2026 and found that combining image and text into one embedding beats either alone by up to 50 per cent, with averaged image and text embeddings putting the correct product in the top spot up to 1.5 times as often as image embeddings alone. The older, narrower CLIP-style model beat the newer, more general one, which was not what the researchers expected.

Title-only embeddings miss critical context, so best practice is to embed a structured concatenation: title plus brand plus category plus key attributes plus first 200 characters of description. Pre-trained models work well for general use cases but may struggle with niche product categories, for example a generic text embedding model might not distinguish between "joggers" (pants) and "joggers" (athletes) without fine-tuning, so if a catalogue includes specialised items such as industrial parts or luxury fashion, fine-tune a base model like SBERT using product data.

How do Algolia and Elasticsearch implement vector search?

Algolia's natural language processing grasps query intent and recognizes synonyms, misspellings, and conversational language, with hybrid search combining vector and keyword capabilities where vector search works for longer queries while keyword search handles exact phrases and single-word searches. Algolia creates semantic connections using vector embeddings, mathematical models that represent words, using tools like Universal Sentence Encoder and sentence transformers. Algolia has the smoothest learning curve with clear documentation and accessible implementation, with teams able to implement search in 2 to 4 weeks instead of 3 to 6 months.

Elasticsearch supports semantic matching through dense vector fields, though the platform needs more technical expertise to work well. Where query complexity exceeds what Algolia's rules engine can handle, custom scoring, complex faceting across nested data structures, or hybrid search combining keyword and vector similarity, Elasticsearch is the right substrate. For vector databases, options include Pinecone, Weaviate, Milvus, or pgvector, and for catalogues under 10 million products, pgvector with IVFFlat or HNSW indexing handles visual search at acceptable latency, whilst for catalogues above 10 million, dedicated vector databases with distributed indexing are required.

What infrastructure choices affect vector search performance?

The architecture decision separates managed services from self-hosted infrastructure. For a 5 million record catalogue with 10 million searches per month, indicative cost for Algolia runs $4,200 to $7,000 per month compared to $1,800 to $3,500 per month on Elastic Cloud, whilst for 50 million records with 100 million searches per month, Algolia costs $35,000 to $60,000 per month versus $9,000 to $18,000 per month on Elastic Cloud. Vendor total cost of ownership data shows $50,000 to $300,000 annually for self-managed infrastructure, plus 1 to 3 dedicated engineers.

An apparel case study saw search exit rates drop by 35 per cent within three months of introducing semantic search. Key performance metrics from 2026 show conversion rate for AI visual search at 5.2 per cent versus 4.1 per cent for standard keyword, revenue per search at $2.80 versus $2.10 for standard, and a conversion rate lift versus traditional search approximately 26 per cent higher. Visual search queries grew 85 per cent year over year in 2025, outpacing every other discovery channel.

How are fashion retailers applying vector search today?

Stylitics' proprietary Vision API extracts visual attributes and indexes every product into a vector database for semantic similarity search. Steve Madden implements AI-powered visual search solutions and personalized product recommendations through Fast Simon's AI technology, an approach that has significantly improved customer experience, boosting engagement and conversion rates through responsive merchandising and trend forecasting. ASOS employs AI-driven visual search and virtual fitting room technologies, significantly improving product discovery and customer confidence, with enhanced customer journeys resulting in increased user engagement, higher repeat visits, and improved conversion rates.

Shopify's data shows nearly half of all shoppers now rely on AI tools for product discovery and inspiration, rising to two-thirds among luxury fashion consumers specifically. According to the State of Fashion 2026 report, shopping-related searches on generative AI platforms grew 4,700 per cent between 2024 and 2025, and of US consumers who used AI for search, 53 per cent also used it to shop, with 85 per cent reporting a better shopping experience through generative AI than traditional search.

What practical steps enable successful implementation?

Implementation begins with model selection aligned to data type and scale requirements. Choose embedding models by evaluating the type of data you need to process and the trade-offs between accuracy, speed, and scalability. Embedding models are typically pretrained on MS MARCO, BEIR or domain-specific e-commerce datasets and can be fine-tuned on shop-specific product language. Domain-specific fine-tuning on product imagery improves retrieval precision for specialised catalogues in fashion, furniture or electronics.

Use approximate nearest neighbour libraries such as FAISS or Annoy to optimize retrieval speed. If product data includes multiple languages, opt for multilingual models like LaBSE or XLM-R. Algolia's synonym management is GUI-driven, which means merchandising teams can handle it without filing engineering tickets, and faceting is pre-computed at indexing time, which means facet counts in navigation are fast without expensive aggregation queries at request time. The shift raises the bar on product data, as if an AI shopping assistant cannot clearly understand what a product is, who it is for and how it fits, it will not get recommended, no matter how good the imagery looks on the product page itself.