How to Evaluate RAG Pipeline Quality
The most common mistake in RAG evaluation is testing with a few hand-picked questions, seeing reasonable answers, and declaring the system ready for production. This approach misses systematic failures: queries where retrieval consistently returns irrelevant chunks, document types that are poorly chunked, edge cases where the model hallucsinates despite having correct context, and knowledge gaps where the answer is not in the knowledge base but the system confidently fabricates one. Proper evaluation requires a structured dataset, quantitative metrics, and the discipline to measure before and after every pipeline change.
Step 1: Build the Evaluation Dataset
The evaluation dataset is a collection of question-answer pairs where each question has a known correct answer and a labeled set of documents (or specific passages within documents) that contain that answer. This dataset is the foundation of all evaluation work, and its quality determines whether your metrics are meaningful.
Creating this dataset is manual work. For each question, you need to: write a natural language question that a real user might ask, write the correct answer (or acceptable answer range), and identify the specific chunks in your knowledge base that contain the information needed to answer. The minimum viable dataset is 50 questions, but 100 to 200 provides more reliable metrics and better coverage of your document types and question categories.
Diversify the dataset across these dimensions: question complexity (simple factual lookups, multi-document synthesis, comparison questions), document types (if your knowledge base contains manuals, FAQs, policies, and reports, include questions from each), answer types (short factual answers, explanatory answers, lists, "not answerable from this knowledge base"), and phrasing (include paraphrased versions of the same question to test retrieval robustness).
Include 10% to 20% of questions whose answers are not in the knowledge base. These test the system's ability to decline rather than hallucinate, which is critical for trust. A system that confidently generates a plausible but fabricated answer to an unanswerable question is more dangerous than one that occasionally fails to answer an answerable question.
You can bootstrap the dataset with LLM assistance: have GPT-4 or Claude read your documents and generate candidate questions, then have a domain expert verify the questions, answers, and relevant chunk labels. This is faster than writing everything from scratch but still requires expert review because the LLM may generate questions that are ambiguous, trivial, or not representative of real user queries.
Step 2: Measure Retrieval Quality
Retrieval metrics tell you whether the system finds the right documents, independent of whether the LLM generates a good answer from them. This separation is essential for diagnosis: if retrieval quality is high but answer quality is low, the problem is in generation (prompt construction, model capability, or system instruction). If retrieval quality is low, no generation improvements will help because the model does not have the right information.
Recall@k is the most important retrieval metric. It measures what fraction of the relevant documents appear in the top k results. Recall@5 tells you how often the correct chunk is in the top 5 results that will go into the prompt (after reranking). A recall@5 of 0.85 means 85% of questions retrieve the right document in the top 5 results, which is a solid baseline for production. Below 0.70, users will notice frequent failures.
Precision@k measures what fraction of the top k results are relevant. High precision means the retrieved results are mostly useful, low precision means the LLM receives a mix of relevant and irrelevant chunks, which can confuse it. For RAG, recall matters more than precision (missing the right document is worse than including an irrelevant one), but very low precision (below 0.30) degrades generation quality because the irrelevant chunks add noise.
Mean Reciprocal Rank (MRR) measures where the first relevant document appears in the ranked results. An MRR of 0.80 means the first correct document is, on average, at position 1.25 (between first and second). MRR is a good summary metric that captures both whether the right document was retrieved and how highly it was ranked.
NDCG@k (Normalized Discounted Cumulative Gain) accounts for the position of every relevant document, not just the first. It penalizes relevant documents that appear lower in the ranking more than those that appear higher. NDCG is the most comprehensive retrieval metric but also the most complex to interpret. Use it alongside recall for a complete picture.
Run these metrics on your full evaluation dataset and segment the results by question type, document type, and complexity. Aggregate metrics can hide systematic failures: overall recall might be 0.85, but recall for questions about pricing tables might be 0.40 because your chunking splits tables incorrectly. Segmented analysis reveals these pockets of poor performance that aggregate metrics obscure.
Step 3: Measure Generation Quality
Generation metrics tell you whether the LLM produces a good answer from the retrieved context. These metrics use a different methodology: instead of comparing retrieved documents against a relevance label, they evaluate the generated text against the context and the question.
Faithfulness (also called groundedness) measures whether every claim in the generated answer is supported by the retrieved context. An answer is unfaithful when it includes information not present in the context, which indicates hallucination. Faithfulness is the most critical generation metric for RAG because the entire point of RAG is to ground responses in retrieved data. A faithfulness score below 0.85 means the model is regularly adding information from its parametric knowledge, which defeats the purpose of the retrieval step.
Answer relevance measures whether the generated answer actually addresses the user's question. An answer can be faithful (every claim is in the context) but irrelevant (it discusses a topic from the context that was not asked about). Low answer relevance usually indicates a prompt construction problem: the system instruction is not clear enough about what the model should focus on, or the retrieved chunks are tangentially related to the question.
Answer completeness measures whether the answer uses all the relevant information from the retrieved context. A complete answer draws from every relevant chunk to provide a thorough response, while an incomplete answer picks up on one chunk and ignores the others. Low completeness often indicates a context formatting problem: chunks are not clearly delineated, or the most relevant chunks are buried where the model's attention is weakest.
Answer correctness compares the generated answer against the reference answer in your evaluation dataset. This is an end-to-end metric that captures both retrieval and generation failures but cannot distinguish between them. Use it as a top-level quality indicator, then drill into retrieval and generation metrics to diagnose root causes when correctness drops.
All four metrics can be computed automatically using LLM-as-judge frameworks. The judge LLM reads the question, the retrieved context, and the generated answer, then evaluates each dimension on a scale (typically 0 to 1). RAGAS, DeepEval, and Trulens are the most widely used frameworks. Each provides slightly different implementations of these core metrics, but the concepts are the same.
Step 4: Run End-to-End Evaluation
Combine retrieval and generation metrics into an end-to-end evaluation run that tests the full pipeline on your evaluation dataset. This is your system's scorecard, the numbers that tell you whether the pipeline is ready for production and whether each change you make improves or degrades quality.
The evaluation loop is: for each question in the dataset, run the full pipeline (retrieval + reranking + generation), record the retrieved chunks and the generated answer, compute retrieval metrics (recall@5, MRR) by comparing retrieved chunks against the labeled relevant chunks, compute generation metrics (faithfulness, relevance, completeness) using the LLM judge, and compute answer correctness by comparing the generated answer against the reference answer.
Aggregate the metrics across the full dataset and segment by question category. The segmented view is where actionable insights come from. Common patterns include: high recall but low faithfulness (retrieval works but the model ignores context), low recall on specific document types (chunking or embedding problem for those documents), high faithfulness but low completeness (the model uses only the first retrieved chunk and ignores the rest), and low correctness on unanswerable questions (the model hallucinates instead of declining).
Run this evaluation before deploying any pipeline change (new chunking strategy, different embedding model, updated system instruction, new reranker). Compare metrics before and after the change. If any metric drops by more than 5%, investigate before deploying. This discipline is the difference between a RAG system that improves over time and one that regresses unpredictably.
Step 5: Set Up Production Monitoring
Evaluation datasets catch known failure modes, but production traffic reveals unknown ones. Set up monitoring that captures quality signals from live queries and alerts you to degradation.
Log everything: Every query, the retrieved chunks (IDs and scores), the generated answer, latency breakdowns (embedding time, retrieval time, reranking time, generation time), and token counts. This log is your debugging tool when issues arise and your data source for improving the evaluation dataset.
User feedback: Add a thumbs up/thumbs down mechanism to every response. This is the most direct quality signal. Track the satisfaction rate (thumbs up / total ratings) over time and set alerts when it drops below your baseline. Segment by query type to identify specific problem areas.
Automated quality checks: Run the faithfulness metric on a sample of production queries (5% to 10%) using the LLM judge. This catches hallucination patterns that users might not report. If the sampled faithfulness score drops below 0.85, trigger an alert for investigation.
Retrieval drift detection: Track the average retrieval score (cosine similarity of the top result) over time. A gradual decline indicates that new user queries are diverging from the knowledge base, which means either the knowledge base needs updating or new document types need to be ingested. A sudden drop indicates a system issue (embedding model change, database corruption, index rebuild failure).
Knowledge base freshness: Track the age distribution of retrieved documents. If the median document age in retrieved results is increasing over time, the knowledge base is becoming stale and needs an update cycle. Set alerts when the median age exceeds a threshold appropriate for your domain (1 month for rapidly changing content, 6 months for stable documentation).
Evaluate retrieval and generation independently. Recall@5 above 0.85 and faithfulness above 0.85 are solid production baselines. Build a diverse evaluation dataset of 50 to 100 questions before you start tuning. Run end-to-end evaluation before every pipeline change. Monitor production quality with user feedback, sampled faithfulness checks, and retrieval drift detection. The evaluation infrastructure is not optional, it is the difference between a system that works and a system you can prove works.