While standard RAG and Corrective RAG (CRAG) improve LLM accuracy, they often suffer from indiscriminate retrieval—fetching documents even for simple questions—and blind trust in retrieved data. Self-Reflective RAG (Self-RAG) solves this by making the AI “conscious” of its own process through constant self-evaluation.
The 3 Core Problems with Traditional RAG #
The sources highlight three major flaws that Self-RAG addresses:
- Unnecessary Retrieval: Systems often fetch data for questions the LLM already knows (e.g., “How many seconds in a minute?”), which leads to computational waste and “unconfident” answers.
- Blind Trust: Standard RAG forces the LLM to answer based on retrieved chunks, even if those chunks are irrelevant or logically mismatched.
- No Self-Verification: Once an answer is generated, traditional RAG doesn’t check if the answer is a hallucination or if it actually answers the user’s question.
The Self-RAG Workflow (Working Diagram) #
The Self-RAG architecture operates as a self-reflective loop across four critical nodes:
- Retain or Retrieve? → Can the LLM answer using its own knowledge? If yes, it generates directly. If no, it retrieves.
- Relevance Filter → Are the retrieved documents actually useful? Irrelevant documents are discarded.
- Hallucination Check (Groundedness) → Is every fact in the generated answer supported by the documents? If not, the answer is revised.
- Usefulness Check → Does the final grounded answer actually solve the user’s query? If not, the query is rewritten and the process repeats.
Implementation with LangGraph #
To build this, we define a comprehensive State to track the reflection scores.
1. Defining the Graph State #
The state tracks whether retrieval is needed, the filtered documents, and the verdicts for support and usefulness.
class GraphState(TypedDict):
question: str
need_retrieval: bool # Yes/No decision
docs: List[str] # Initial retrieved docs
relevant_docs: List[str] # Filtered relevant docs
context: str # Merged relevant text
answer: str # Generated response
is_support: str # 'fully', 'partially', or 'no support'
is_use: str # 'useful' or 'not useful'
retrieval_query: str # Rewritten query if needed
2. The Decision Node: Should We Retrieve? #
This node forces the LLM to output a JSON schema (True/False) based on whether the question requires specific external facts.
# Logic for the 'Decide Retrieval' Node
def decide_retrieval(state):
# System Prompt: "True if external docs are needed, else False."
# If the question is general (e.g., "What is AI?"), choose False.
# If it's specific (e.g., "Company refund policy"), choose True.
return {"need_retrieval": decision}
3. The Hallucination (Support) Node #
This is the heart of Self-RAG. It evaluates if the generated answer is grounded in the context.
- Fully Supported: All facts are in the docs.
- Partially Supported: Some facts are fabricated by the LLM’s own memory.
- No Support: The answer is a complete hallucination.
def check_hallucination(state):
# Inputs: Question, Answer, Context
# Output: 'fully_supported', 'partially_supported', or 'no_support'
if verdict == 'partially_supported':
return "revise" # Routes to a Revision Node
return "next_step"
4. The Usefulness & Query Rewrite Node #
If the answer is grounded but not useful (e.g., the documents didn’t have the specific detail needed), the system rewrites the query for a better vector search.
def rewrite_query(state):
# System Prompt: "Rewrite the user's question into a
# query optimized for vector retrieval."
return {"retrieval_query": new_query}
Conclusion #
Self-RAG transforms RAG from a linear process into a dynamic, self-correcting agent. By checking for relevance, hallucinations, and final usefulness, it ensures that the user receives the most accurate and grounded information possible, while avoiding the pitfalls of irrelevant data and AI “guessing
Q.1 What is the primary characteristic that distinguishes Self-RAG from traditional RAG architectures?
A fixed linear flow that prevents looping or revising previously generated answers.
The ability of the LLM to actively judge its own retrieval evidence and generated responses.
The exclusion of an LLM's parametric knowledge during the generation process.
The use of multiple vector databases for simultaneous retrieval.
Explanation
Self-RAG extends traditional RAG by enabling the LLM to evaluate the quality of retrieved documents and its own generated answers, allowing iterative retrieval, revision, and self-reflection.
Q.2 Which term is used in the source to describe the flaw where a RAG system retrieves documents regardless of the simplicity of the query?
Parametric Hallucination
Semantic Overlap
Indiscriminate Retrieval
Redundant Vectorization
Explanation
Traditional RAG often performs retrieval for every query, even when unnecessary. This behavior is called ‘Indiscriminate Retrieval’ because the system lacks the ability to decide whether retrieval is actually needed.
Q.3 In the Self-RAG architecture, what is the purpose of the 'Is Relevant' node?
To decide whether the LLM's parametric knowledge is sufficient to answer the question.
To filter out retrieved document chunks that do not provide useful information for the specific query.
To verify if the final generated answer is grammatically correct.
To check if the user's question contains hate speech or toxic content.
Explanation
The ‘Is Relevant’ node evaluates each retrieved document chunk and removes irrelevant ones so that only useful context is passed to the answer generation step.
Q.4 According to the source, how is a 'partially supported' answer defined during the hallucination check?
An answer where some facts come from the retrieved documents but others are fabricated by the LLM.
An answer where the model only retrieves half of the required documents.
An answer that is technically correct but uses a different language than the retrieved documents.
An answer that identifies the correct document but fails to extract the specific answer.
Explanation
A partially supported answer contains information grounded in the retrieved documents but also includes unsupported or fabricated details introduced by the LLM, making it only partially trustworthy.
Q.5 What occurs in the Self-RAG workflow if a generated answer is found to be 'Fully Supported' but 'Not Useful'?
The system presents the answer to the user anyway because it is factually grounded.
The system triggers a 'Query Rewrite' to perform a new, optimized retrieval.
The system ignores the documents and generates a new answer based solely on parametric knowledge.
The system immediately terminates the flow and reports a system error.
Explanation
If an answer is factually correct but fails to answer the user’s question effectively, Self-RAG rewrites the query and performs another retrieval to gather better supporting information.
Q.6 When building a Self-RAG system in LangGraph, why are 'Max Retries' implemented in the revision and retrieval loops?
To bypass the vector store and use web search exclusively after three attempts.
To prevent the system from entering an infinite loop that consumes excessive tokens or time.
To ensure the LLM tries at least five different writing styles before finishing.
To satisfy API rate limits imposed by the LLM provider.
Explanation
Max Retries provide a safety mechanism that limits repeated retrieval and revision attempts, preventing infinite loops, excessive token usage, and unnecessary computation.
Q.7 Which of the following scenarios would likely result in the 'Need Retrieval' decision being 'False' in a Self-RAG system?
A user asks for a comparison of three different internal product manuals.
A user asks for the current stock price of a specific company.
A user asks for the refund policy of a private company like Nexa AI.
A user asks for a general definition of a common concept like 'What is a minute?
Explanation
Simple factual questions that are well covered by the LLM’s parametric knowledge, such as defining a minute, do not require document retrieval, so the system can answer directly.
Q.8 What is the specific role of the 'Revise Answer' node when a response is flagged for hallucinations?
It deletes the answer and asks the user to rephrase their question.
It increases the LLM temperature to encourage more creative writing.
It performs a web search to find evidence supporting the hallucinated claims.
It uses a strict prompt to remove any claims that are not supported by the retrieved context.
Explanation
The ‘Revise Answer’ node rewrites the response by removing unsupported statements and ensuring every claim is grounded in the retrieved evidence.
Q.9 In the implementation discussed, what does the system use as the 'Retrieval Query' during its first attempt to fetch documents?
A summary of the LLM's parametric knowledge.
A randomly selected keyword from the company profile.
The original question provided by the user.
A list of five high-signal keywords generated by a Rewrite node.
Explanation
The initial retrieval query is simply the user’s original question. Only if retrieval fails or produces poor results does the system rewrite the query for subsequent attempts.
Q.10 Why does the Self-RAG architecture check for 'Usefulness' after checking for 'Hallucination'?
Because the 'Is Useful' node automatically fixes hallucinations.
Because checking usefulness is computationally cheaper than checking hallucinations.
Because usefulness is only relevant for parametric answers.
Because an answer must first be factually correct before determining whether it is helpful.
Explanation
Self-RAG first ensures that an answer is fully grounded in the retrieved evidence. Only after verifying factual correctness does it evaluate whether the answer is actually useful and satisfies the user’s request.
