Suppose a hypothetical support assistant can retrieve conceptually related articles but fails when users mention an error code, invoice number, API method, or new product name. The embedding search returns passages about the general topic while missing the document containing the exact identifier.
Vector search misses exact IDs and product names. How would you improve retrieval?
This behavior is expected. Semantic similarity is useful for paraphrases and concepts, but exact strings and rare tokens may not dominate an embedding representation.
Why the common answer is incomplete
Most candidates answer: “Use a better embedding model.”
A different model may improve a benchmark, but it does not remove the mismatch between semantic retrieval and exact-match intent. Identifiers can be arbitrary, newly created, case-sensitive, or absent from the model’s training distribution. Embedding upgrades also require re-indexing and may still underperform a lexical index on the cases that matter.
A production retrieval system combines complementary signals and evaluates them by query type.
A production approach
1. Build a query taxonomy from failures
Sample real searches and classify them into exact identifiers, named entities, natural-language questions, error messages, and mixed queries. Record which relevant document should have been retrieved and why the current system missed it. The taxonomy determines whether the problem is tokenization, filtering, freshness, chunking, semantic ranking, or an authorization rule—not every miss needs another retriever.
2. Normalize without destroying meaningful syntax
Create searchable forms for case, Unicode, punctuation, and common separators, while retaining the original value. Product codes such as AB-123, method names, and version strings may depend on punctuation. Store dedicated keyword fields for identifiers extracted during ingestion. Query both the original and normalized form so cleanup does not turn distinct identifiers into the same token.
3. Retrieve lexical and semantic candidates in parallel
Use a lexical index for exact terms, rare tokens, and phrase matches, and a vector index for semantic similarity. Apply tenant, document-type, and permission filters consistently to both paths. Allocate candidate counts by query class rather than using one fixed value. An identifier-heavy query may need more lexical candidates, while a conceptual question may rely more heavily on semantic retrieval.
4. Fuse scores without assuming they share a scale
Lexical and vector scores have different meanings and distributions. Avoid adding raw scores directly. Use rank-based fusion, calibrated scores, or a learned combination trained on representative judgments. Preserve the source rank and matching terms in the trace. Fusion should allow a strong exact match to survive even when semantic candidates are broadly related.
5. Rerank with the full query and useful metadata
Pass a bounded candidate set to a reranker that considers the complete query, title, passage, document type, recency, and exact-match features. Keep deterministic boosts for authoritative or current documents when product requirements justify them. The reranker cannot rescue a relevant document that neither first-stage retriever returned, so measure candidate recall before tuning final ranking.
6. Choose chunks that preserve identifiers and context
Avoid separating an error code from its explanation or a method signature from its description. Use structure-aware chunking for tables, API references, headings, and code. Add document title and section path to the indexed representation. When an identifier maps to a small fragment, retrieve neighboring content so the generator receives enough context to answer rather than only the matching token.
7. Handle zero-result and ambiguity cases deliberately
If an exact identifier has no authorized match, say so rather than substituting a semantically similar product. Offer a corrected spelling only when confidence is high. When several versions match, prefer the version allowed by the user’s environment or ask for clarification. A retrieval system should distinguish “nothing found” from “many plausible results” because the safe generation behavior differs.
8. Evaluate by query slice
Measure first-stage recall, reranked recall, reciprocal rank, answer groundedness, and no-result accuracy separately for identifiers, entities, conceptual questions, and mixed queries. Include unseen identifiers and newly launched products. Track whether an improvement in exact matching crowds out semantic results. The target is not one global retrieval score; it is reliable behavior across the query distribution.
Engineering trade-offs
Running two retrievers increases infrastructure, latency, and operational tuning. Larger candidate pools improve recall while making reranking slower and more expensive. Aggressive exact-match boosts help identifiers but can overvalue incidental strings copied into unrelated documents. Structure-aware chunking improves precision but requires source-specific parsers and re-indexing.
A learned fusion model can outperform hand-tuned rules, yet it needs trustworthy labels and drift monitoring. Simple rank fusion is easier to operate and often a good baseline. The choice should follow measured failure modes rather than a preference for a particular search technology.
Common mistakes
- Expecting embeddings to behave like an exact keyword index.
- Lowercasing or stripping punctuation from identifiers without preserving the original.
- Combining raw lexical and vector scores as if they were comparable.
- Tuning a reranker before measuring whether relevant documents enter the candidate set.
- Applying authorization filters to one retrieval path but not the other.
- Returning a semantically similar result when an exact identifier has no match.
- Evaluating only conversational questions and excluding codes, versions, and method names.
Failure handling and observability
Trace normalized query forms, detected query class, filters, candidate ranks from each retriever, fusion contribution, reranker score, selected passages, and answer citations. Monitor no-result rate, exact-match recall, candidate overlap, reranker latency, index freshness, and results suppressed by permissions. Review failed searches with tenant-safe identifiers and feed confirmed cases into the evaluation set.
If the vector index is unavailable, serve lexical results for eligible queries and disclose reduced semantic capability. If the lexical path fails, semantic retrieval can continue for conceptual questions but should not pretend to verify an exact code. When the reranker times out, fall back to a deterministic fused rank. Keep each degradation mode explicit so operators can distinguish lower relevance from a complete search outage.
Interview insight
The interviewer is not testing whether you know a particular vector database. They are testing whether you can match retrieval mechanisms to query intent. A staff-level design recognizes that exact and semantic relevance are complementary, fuses them deliberately, and measures recall before asking generation to compensate for missing evidence.