How to Build a Hybrid Search System (Playbook)

Pure vector search fails on the queries enterprises care about most: part numbers, policy names, error codes, people. Pure lexical search fails on the queries users actually type: questions in their own words. Hybrid search runs both and merges them, and it is the retrieval foundation every serious RAG or enterprise search system should rest on. This playbook covers the build, following FISTA's AI enablement practice. Concepts are in what is hybrid search, what is bm25, and what is semantic search.

What does the system do?

Stage	Function
Ingestion	One chunking, metadata, and permission model feeding both indexes
Lexical index	BM25 or equivalent over chunk text with analyzers
Vector index	Embeddings per chunk in a vector search service
Query processing	Rewriting, expansion, spelling, type detection
Parallel retrieval	Both indexes with shared filters and permissions
Fusion	Reciprocal rank fusion or learned combination
Reranking	Cross-encoder or model reranker on top candidates
Evaluation	Component-wise relevance metrics by query type

Step 1: Analyze queries and define success

Collect real queries and classify them: exact identifier, named entity, short keyword, natural-language question, conceptual. Build a labeled set with graded relevance per query. Define latency and freshness targets. The query mix determines fusion weights and evaluation priorities. See how to write acceptance criteria for ai.

Step 2: Ingest once, index twice

Parse and chunk content once with structure awareness, attach metadata and permissions, and write each chunk to both indexes with a shared identifier. Divergent chunking between indexes makes fusion incoherent. Ingestion design is in the enterprise RAG reference architecture whitepaper.

Step 3: Build the lexical index

Configure analyzers for your languages and content: tokenization, stemming or lemmatization, stopword handling, and preservation of identifiers and codes as exact tokens. Boost fields such as titles. Test on identifier and name queries specifically.

Step 4: Build the vector index

Embed chunks with a model chosen by evaluation and pinned; store in a vector search service with the same metadata and permission attributes. See how to build a vector search service.

Step 5: Process queries

Apply spelling correction, abbreviation expansion, and rewriting with conversation context; detect query type to adjust fusion weights; generate lexical and vector query forms. Concepts are in what is query rewriting.

Step 6: Retrieve in parallel and fuse

Run lexical and vector retrieval concurrently with identical filters and permission constraints applied before ranking; take the top candidates from each; fuse with reciprocal rank fusion, weighted by query type where evaluation supports it, or with a learned combiner trained on your labeled set.

Step 7: Rerank

Apply a cross-encoder or model reranker to the fused top candidates and cut to the result count the consumer needs. Bound the candidate set to control latency. Concepts are in what is a reranker and what is a cross-encoder.

Step 8: Evaluate component-wise

On the labeled set, measure recall and precision at k and normalized discounted cumulative gain for lexical alone, vector alone, fused, and reranked, by query type. Tune analyzers, embedding choice, fusion weights, and reranker cutoffs from the results. Track click and reformulation signals in production and add poorly served queries to the set. Method is in the AI evaluation and testing whitepaper.

Worked example: a manufacturer's technical knowledge base

A manufacturer's engineers search manuals, service bulletins, and part catalogs. Query analysis shows a mix of part numbers, fault codes, and natural-language troubleshooting questions. Ingestion chunks documents by section with part and model metadata and feeds both indexes. The lexical analyzer preserves part numbers and fault codes as exact tokens; the vector index uses an embedding model that evaluation shows handles technical prose well. Query processing detects identifier queries and weights lexical results higher for them. Component-wise evaluation shows lexical search alone dominating identifier queries, vector search alone dominating troubleshooting questions, fusion improving both, and reranking lifting top-three precision across the board. The system becomes the retrieval layer for a grounded troubleshooting assistant, whose answer quality tracks the retrieval metrics.

What does it cost to run?

Cost is two indexes plus reranking per query; reranking is the largest per-query cost and is bounded by candidate cutoffs. Value is measured in relevance gains and, downstream, in RAG answer quality. Drivers are in vector database cost and enterprise rag cost.

What are the common mistakes?

• Different chunking for each index.
• Analyzers that stem part numbers into nonsense.
• Applying permissions after fusion.
• Skipping reranking to save latency.
• One fusion weight for every query type.
• Evaluating the final list only, never the components.

How do you phase the build?

Ship lexical search first with tuned analyzers and the labeled query set, because it is cheap, fast, and already better than most incumbent search for identifier queries. Add the vector index and fusion next and measure the gain on natural-language queries. Add reranking last and confirm its precision gain justifies its latency. Each stage is measured component-wise, so the team knows what each addition bought and can defend the architecture.

What team does it need?

A search engineer who owns analyzers, indexes, and fusion, a data or ML engineer for embeddings and reranking, and the content owners who label queries and judge relevance. Query labeling by people who know the domain is the work most teams skip and most regret skipping.

How do you handle multilingual content?

Lexical indexes need language-aware tokenization and stemming per language, embeddings need a model evaluated on the languages in the corpus, and queries should be detected and routed accordingly. Cross-language retrieval, where a query in one language should find documents in another, relies on multilingual embeddings and deserves its own golden set, because lexical matching contributes little there.

How FISTA Solutions builds hybrid search

FISTA Solutions builds hybrid search systems to this playbook: query-typed specifications, single ingestion feeding lexical and vector indexes, tuned analyzers, evaluated embeddings, query processing, parallel retrieval with shared permissions, weighted fusion, bounded reranking, and component-wise evaluation. The AI enablement practice delivers the retrieval layer, AI agents build grounded assistants on it, and forward deployed engineers embed with your content teams to build the labeled query set. The record behind the work is 150+ projects with 99.9% uptime.

To scope a hybrid search system, message FISTA on WhatsApp, or read how to build a semantic search engine for the full search product.