Skip to main content

Chapter-1.4---Evaluation & System Design

Q8: How do you systematically evaluate a RAG pipeline using the RAG Triad?​

Answer: Evaluating RAG requires separating retrieval performance from generation performance. The RAG Triad framework measures three core metrics:

[ User Query ] / \


Context / \ Groundedness

Relevance/

▼ ▼

[ Context ] ──────> [ Response ]

Answer

Relevance

  1. Context Relevance: Does the retrieved context actually contain the information needed to answer the user query? (Evaluates Retriever)
  2. Groundedness (Faithfulness): Is the generated response supported entirely by the retrieved context, without hallucinating outside facts? (Evaluates LLM Generation)
  3. Answer Relevance: Does the generated response directly address the user's original query? (Evaluates End-to-End Output)

Q9: How do you handle unanswerable queries or out-of-domain questions?​

Answer:

  • Distance Thresholding: Set a minimum cosine similarity threshold. If retrieved chunks fall below the score, trigger a fallback mechanism ("I don't have enough context to answer that.").
  • Self-RAG / Corrective RAG (CRAG): Implement an evaluator node that checks whether retrieved documents are relevant before generating a response.
  • Explicit Prompt Framing: Instruct the model: "Answer the user query ONLY using the provided facts below. If the information is missing, explicitly state that you do not know."

Q10: What causes poor retrieval quality in RAG pipelines?​

Answer:

  1. Bad Chunking: Chunks that are too small lack context, while chunks that are too large dilute the semantic meaning.
  2. Sub-optimal Embedding Models: Using generic models (like standard text-embedding-ada) for highly specialized domain vocabulary (like medical or legal terms) without fine-tuning.
  3. Over-reliance on Dense Search: Failing to catch exact keyword matches or serial numbers because sparse retrieval (BM25) wasn't used in a hybrid setup.
  4. No Reranking: The vector database returns broadly related documents, but the most critically relevant chunk might be ranked #15 instead of #1.

Q11: How would you reduce hallucinations in a RAG system?​

Answer:

  1. Strict System Prompts: Enforce rules like "Answer ONLY using the provided context."
  2. Chain-of-Verification / Self-Correction: Have the model evaluate its own output against the retrieved chunks before returning the final response.
  3. Citation Enforcement: Require the model to cite exact document IDs or line numbers for every claim it makes.
  4. Context Truncation: Remove low-relevance chunks from the prompt, as injecting irrelevant information drastically increases the hallucination rate.

Q12: How would you optimize vector database search latency?​

Answer:

  1. Approximate Nearest Neighbor (ANN): Ensure you are using ANN indexes like HNSW (Hierarchical Navigable Small World) or IVF-PQ rather than exhaustive K-Nearest Neighbor (KNN) search.
  2. Metadata Filtering: Apply pre-filtering (e.g., date > 2023, category = "finance") before executing the vector search to drastically reduce the search space.
  3. Dimensionality Reduction: Use embedding models with lower dimensions or apply PCA to reduce vector size.
  4. Quantization: Store vectors in INT8 or binary formats (like Cohere's binary embeddings) to speed up distance calculations and reduce memory bandwidth.

Q13: How would you handle document updates without rebuilding the entire index?​

Answer:

  1. Document ID Tracking: Assign a unique ID to every parent document and propagate it to all its child chunks as metadata.
  2. Upsert Operations: When a document updates, first issue a DELETE operation to the vector database for all chunks matching that document_id.
  3. Re-embedding: Process the updated document, generate new chunks, embed them, and INSERT them into the database.
  4. Soft Deletes: If the database doesn't support fast deletes, mark the old document IDs as "inactive" in a fast relational database (like Redis or Postgres), and filter them out during the retrieval step.

Q14 :​

Answer:

  1. **Bad