In the world of Generative AI, Retrieval-Augmented Generation (RAG) has become the standard for grounding LLMs in private data. However, traditional RAG suffers from a “blind trust” problem where the system assumes retrieved documents are always relevant. If the retriever fetches incorrect or irrelevant context, the LLM will likely hallucinate or provide a wrong answer.
Corrective RAG (CRAG) is an advanced strategy designed to solve this by adding a self-correction layer that evaluates the quality of retrieved documents before they reach the generation stage
The CRAG Workflow: How It Works #
CRAG introduces a Retrieval Evaluator between the retrieval and generation steps. This evaluator analyzes the relationship between the user query and the retrieved documents to decide on one of three paths:
- Correct: If the documents are highly relevant, the system performs Knowledge Refinement to remove noise and then generates the answer.
- Incorrect: If the documents are irrelevant, the system triggers a Web Search (e.g., using Tavily) to find the correct information externally.
- Ambiguous: if the documents are partially relevant, the system combines the “good” parts of the internal documents with external web search results for a comprehensive answer.
The Working Diagram #
Key Concept: Knowledge Refinement #
Even when documents are “Correct,” they often contain “garbage” or irrelevant text due to fixed-size chunking. CRAG uses a three-step refinement process:
- Decomposition: Breaking the document into individual sentences or “strips”.
- Filtration: Using an LLM to score each strip and keep only those that directly answer the query.
- Re-combination: Merging the kept strips into a clean “Refined Context”.
Implementation with LangGraph #
To implement this, we define a State to track the query, documents, and evaluation results.
1. Defining the State #
from typing import List, TypedDict
class GraphState(TypedDict):
question: str
documents: List[str]
good_docs: List[str] # Docs scoring above lower threshold
wordict: str # 'correct', 'incorrect', or 'ambiguous'
refined_context: str
answer: str
2. The Retrieval Evaluator Node #
We use an LLM to score each document between 0 and 1. We set an Upper Threshold (0.7) for “Correct” and a Lower Threshold (0.3) to filter out useless chunks.
def evaluate_documents(state):
question = state["question"]
documents = state["documents"]
# Logic: Score each doc.
# If any doc > 0.7 -> verdict = 'correct'
# If all docs < 0.3 -> verdict = 'incorrect'
# Otherwise -> verdict = 'ambiguous'
# Filter only docs > 0.3 into 'good_docs'
return {"wordict": verdict, "good_docs": good_docs}
3. Knowledge Refinement Node #
This node cleans the “good documents” by filtering them at a sentence level.
def refine_documents(state):
# 1. Decompose docs into sentences (strips)
# 2. Use LLM to filter strips relevant to the question
# 3. Join relevant strips into 'refined_context'
return {"refined_context": refined_context}
4. The Query Rewrite & Web Search Node #
When the system realizes the internal database lacks information, it rewrites the query for a search engine and fetches new data.
def web_search(state):
# Rewrite the query (e.g., adding "last 30 days" for recency)
# Search via Tavily API
# Add web results to documents list
return {"documents": web_docs}
Summary of Benefits #
By implementing CRAG, your RAG system becomes significantly more robust. It stops blindly trusting its own database and gains the ability to:
- Self-Correct: Identify when its own knowledge is insufficient.
- Reduce Noise: Clean up retrieved text to improve generation quality.
- Augment Dynamically: Use the internet to fill in gaps for recent or niche topics.
This multi-step approach ensures that the final answer is always grounded in the most relevant and refined information available, whether it’s internal or external.
| GitHub Full Code | Click Here |
| Follow Me LinkedIn | Click Here |
CRAG Quiz #
Q.1 What is the fundamental limitation of traditional Retrieval-Augmented Generation (RAG) that Corrective RAG (CRAG) is designed to address?
The lack of support for multi-modal data like images and audio.
The LLM's blind trust in retrieved documents, even if they are irrelevant.
The inability of vector databases to store private documents.
The high computational cost of embedding models.
Explanation
Traditional RAG assumes retrieved documents are relevant. CRAG introduces a Retrieval Evaluator to assess document quality before generation, reducing hallucinations caused by poor retrieval.
Q.2 In the CRAG framework, what is the specific role of the Retrieval Evaluator component?
To fine-tune the LLM on the specific private dataset.
To rewrite the user's query into a format optimized for search engines.
To assess the quality of retrieved documents and determine if they are useful for answering the query.
To convert user queries into high-dimensional vectors.
Explanation
The Retrieval Evaluator scores the retrieved documents and determines whether they are sufficiently relevant for answering the user’s question.
Q.3 During the Knowledge Refinement process, what occurs during the Decomposition step?
The user query is decomposed into multiple sub-queries.
The vector database is split into smaller shards for faster searching.
The LLM's weights are decomposed to reduce model size.
The retrieved document is broken down into smaller units, such as individual sentences or strips.
Explanation
Knowledge Refinement decomposes retrieved chunks into smaller strips, allowing the system to keep only the most relevant information while discarding noise.
Q.4 Which threshold logic is typically used to classify a retrieval as Correct in the CRAG architecture?
All retrieved documents must have a score of exactly 1.0.
The average score of all retrieved documents is above 0.5.
At least one retrieved document has a relevance score higher than the upper threshold (e.g., 0.7).
The sum of all document scores exceeds a predefined limit.
Explanation
If at least one retrieved document exceeds the upper relevance threshold, the retrieval is considered sufficiently accurate for direct answer generation.
Q.5 If the Retrieval Evaluator returns a verdict of Incorrect, what is the next logical step in the CRAG workflow?
The system re-runs the semantic search with a different embedding model.
The system ignores the retrieved documents and performs a web search for external knowledge.
The system tells the user I don't know and ends the session.
The system asks the user to provide more documents.
Explanation
When retrieval quality is poor, CRAG supplements or replaces internal knowledge with external web search results to improve answer quality.
Q.6 What happens in the Ambiguous case in CRAG?
The system automatically lowers its relevance thresholds to zero.
The system randomly selects between internal documents and web search.
The system combines Good Docs from internal retrieval with external knowledge from a web search.
The system prompts the user to clarify their question before proceeding.
Explanation
In ambiguous situations, CRAG merges high-quality internal documents with relevant web search results before generating the final response.
Q.7 Why is Query Rewriting used specifically before the web search component in CRAG?
To shorten the query to save API tokens.
To transform vague or conversational user queries into keyword-rich search terms for search engines.
To translate the query into different languages.
To encrypt the query before sending it to the web.
Explanation
Query Rewriting converts conversational questions into optimized search-engine queries, improving the relevance of web search results.
Q.8 In the implementation code provided, what constitutes a Good Doc during the evaluation phase?
A document that has been successfully summarized by the LLM.
A document chunk with an evaluation score greater than the lower threshold (e.g., 0.3).
Any document containing the exact query keywords.
The first document returned by the vector search.
Explanation
Documents whose evaluation scores exceed the lower threshold are considered Good Docs and are kept for further refinement and answer generation.
Q.9 Why does CRAG perform Knowledge Refinement after document retrieval?
To reduce the size of the vector database.
To remove irrelevant or low-quality information from retrieved documents before generation.
To retrain the embedding model using retrieved documents.
To convert retrieved text into SQL queries.
Explanation
Knowledge Refinement filters out noisy or irrelevant portions of retrieved documents so that only high-quality context is passed to the LLM, improving response accuracy.
Q.10 What is the final step in the Knowledge Refinement process before passing context to the generator?
Translating the strips into the LLM's native language.
Applying a secondary embedding step to the refined text.
Summarizing the documents into a single sentence.
Recombining the filtered kept strips into a refined context string.
Explanation
After filtering relevant strips, CRAG recombines them into a refined context that is supplied to the LLM for generating a more accurate response.
