Skip to main content

Chapter-1.2---Search & Retrieval Engineering

Q4: What is Hybrid Search, and why is Dense Vector Search alone often insufficient?​

Answer: Dense retrieval (vector similarity using cosine distance or dot product) excels at capturing semantic intent and broad concepts, but often struggles with exact keyword matching, specific serial numbers, acronyms, or rare terms.

Hybrid Search combines two complementary retrieval paradigms:

  1. Dense Retrieval (Semantic): Neural network embeddings (e.g., text-embedding-3-large, bge-large-en).
  2. Sparse Retrieval (Keyword): Algorithmic keyword matching like BM25 or TF-IDF.

Score Combining Algorithm (RRF)​

Results from both retrievers are merged using Reciprocal Rank Fusion (RRF):

RRF_Score(d)=∑m∈M1k+rm(d)RRF\_Score(d) = \sum_{m \in M} \frac{1}{k + r_m(d)}

Where rm(d)r_m(d) is the rank of document dd in retriever mm, and kk is a smoothing constant (typically 60).

Q5: What is Re-ranking (Cross-Encoders), and where does it fit in the pipeline?​

Answer: Bi-encoders (standard vector embeddings) process queries and documents independently to generate vectors for fast approximate nearest neighbor (ANN) search. However, they lose subtle query-document interactions.

A Re-ranker (Cross-Encoder) takes the top NN candidates (e.g., top 50) from the initial vector/hybrid search and passes the query and chunk together through a transformer layer to score fine-grained relevancy.


[Query] + [Vector Store] ──> Top 50 Chunks (Bi-Encoder)

│

▼

[Top 50 Chunks] ───────────> [Cross-Encoder Re-ranker] ──> Top 5 Chunks ──> [LLM Prompt]

  • Why it matters: Drastically improves precision and reduces the "lost in the middle" phenomenon without the computational cost of running a cross-encoder across millions of vector database records.