Introduction #

Traditional AI chatbots can answer general questions, but they struggle with private documents, recent information, and factual accuracy. Modern AI applications are moving toward Agentic AI, where an AI agent can reason, make decisions, and use external tools whenever required.

GitHub Link with UI Code	Click Here
Follow Me LinkedIn	Click Here

One of the most important tools in an AI agent is Retrieval-Augmented Generation (RAG).

Instead of depending entirely on the Large Language Model’s training data, a RAG system retrieves relevant information from external knowledge sources such as PDFs, databases, or company documents before generating a response.

In this tutorial, we’ll build a Multi-Utility RAG Chatbot using LangGraph that allows users to:

• Upload PDF documents
• Ask questions about those documents
• Retrieve the most relevant information automatically
• Generate accurate answers grounded in the uploaded files
• Use RAG as just another tool inside an Agentic AI workflow

By the end of this article, you’ll understand how LangGraph enables an AI agent to intelligently decide when it should search a document and when it should simply answer using its own knowledge.

---

Why Do We Need RAG? #

Although modern LLMs such as GPT-4o, Claude, and Gemini are extremely capable, they still suffer from several important limitations.

1. Knowledge Cutoff #

Every LLM is trained on data available only until a certain date.

This means the model has no knowledge of:

• today’s news
• recent research papers
• updated laws
• latest documentation
• newly released software

RAG solves this problem by retrieving information from external sources before generating the answer.

---

2. Private Documents #

Suppose you have:

• company financial reports
• internal documentation
• medical records
• research papers
• personal notes
• university lecture PDFs

These documents are not part of the LLM’s training data.

A normal chatbot cannot answer questions about them.

RAG securely connects your private documents to the language model without requiring model retraining.

---

3. Hallucination #

LLMs occasionally produce incorrect answers while sounding highly confident.

This phenomenon is called hallucination.

Instead of relying only on the model’s memory, RAG provides relevant context from trusted documents.

The model is therefore encouraged to answer using retrieved evidence rather than inventing facts.

---

What is Retrieval-Augmented Generation (RAG)? #

Retrieval-Augmented Generation combines two systems:

• Retriever → Finds relevant information
• Generator (LLM) → Produces the final response

Instead of asking:

“Answer this question.”

we ask:

“Here is the relevant context. Now answer using only this information.”

This dramatically improves factual accuracy.

---

The Core Principle: In-Context Learning #

RAG works because of In-Context Learning.

Rather than permanently changing the model, we temporarily provide additional information inside the prompt.

For example:

Question:
What is Gradient Descent?

Relevant Context:
Gradient Descent is an optimization algorithm...

Answer:
...

The model now answers using both:

• its pretrained knowledge
• the retrieved context

---

Why Can’t We Send the Entire PDF? #

Large Language Models have a limited context window.

Examples:

• 8K tokens
• 32K tokens
• 128K tokens

A 300-page PDF may contain hundreds of thousands of tokens.

Sending the complete document with every query would:

• increase cost
• increase latency
• exceed the model’s context limit

Instead, RAG retrieves only the most relevant sections.

---

RAG Architecture #

---

Step 1: Loading the PDF #

We first load the PDF using LangChain’s PyPDFLoader.

from langchain_community.document_loaders import PyPDFLoader

loader = PyPDFLoader("intro-to-ml.pdf")
docs = loader.load()

print(len(docs))

Each page becomes a Document object containing:

• page content
• page number
• metadata

---

Step 2: Splitting Large Documents #

Large pages are divided into smaller chunks.

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200
)

chunks = splitter.split_documents(docs)

Why overlap?

Chunk 1
------------------------
Machine Learning is...

Chunk 2
            continues...

The overlap preserves context between neighboring chunks.

---

Step 3: Creating Embeddings #

Text cannot be searched efficiently.

Instead, every chunk is converted into a numerical vector.

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="text-embedding-3-small"
)

Embeddings capture semantic meaning.

For example:

Dog
↓

[0.13, -0.25, 0.71, ...]

Cat
↓

[0.12, -0.24, 0.69, ...]

Similar meanings produce similar vectors.

---

Step 4: Storing Embeddings in FAISS #

Now we create a vector database.

from langchain_community.vectorstores import FAISS

vector_store = FAISS.from_documents(
    chunks,
    embeddings
)

FAISS stores vectors for efficient similarity search.

---

Step 5: Creating a Retriever #

The retriever searches the vector database.

retriever = vector_store.as_retriever(
    search_type="similarity",
    search_kwargs={"k":4}
)

Whenever a user asks a question:

Question

↓

Embedding

↓

Similarity Search

↓

Top 4 Relevant Chunks

Only those chunks are passed to the LLM.

---

Step 6: Turning RAG into a LangGraph Tool #

Instead of calling the retriever directly, we wrap it as a tool.

from langchain_core.tools import tool

@tool
def rag_tool(query):
    """
    Retrieve relevant information from the PDF document.
    """

    result = retriever.invoke(query)

    context = [doc.page_content for doc in result]
    metadata = [doc.metadata for doc in result]

    return {
        "query": query,
        "context": context,
        "metadata": metadata
    }

This makes retrieval available to the AI agent whenever needed.

---

Why Implement RAG as a Tool? #

Imagine your AI assistant also has:

• Calculator
• Weather API
• Web Search
• Stock Price API
• MCP Server
• SQL Database
• PDF Search

The agent should decide which tool is appropriate.

For example:

User: What is today’s Tesla stock price? Use: Stock Tool
User: Explain KNN from my uploaded notes. Use: RAG Tool
This is precisely what LangGraph enables.

---

Step 7: Binding Tools to the LLM #

tools = [rag_tool]

llm_with_tools = llm.bind_tools(tools)

Now the model knows that a retrieval tool exists and can call it when appropriate.

---

Step 8: Defining the Graph State #

class ChatState(TypedDict):
    messages: Annotated[
        list[BaseMessage],
        add_messages
    ]

The state stores the ongoing conversation history.

---

Step 9: Creating the Chat Node #

def chat_node(state):

    messages = state["messages"]

    response = llm_with_tools.invoke(messages)

    return {
        "messages":[response]
    }

This node is responsible for:

• understanding user intent
• deciding whether to call a tool
• generating responses

---

Step 10: Creating the Tool Node #

tool_node = ToolNode(tools)

Whenever the LLM requests a tool, this node executes it.

---

Step 11: Building the LangGraph Workflow #

graph = StateGraph(ChatState)

graph.add_node("chat_node", chat_node)
graph.add_node("tools", tool_node)

graph.add_edge(START, "chat_node")

graph.add_conditional_edges(
    "chat_node",
    tools_condition
)

graph.add_edge(
    "tools",
    "chat_node"
)

chatbot = graph.compile()

Workflow:

          START
             │
             ▼
       Chat Node
             │
   Tool Needed?
      /      \
    Yes      No
    │          │
    ▼          ▼
 Tool Node   Final Answer
    │
    ▼
 Chat Node
    │
    ▼
 Final Answer

---

Running the Chatbot #

result = chatbot.invoke(
    {
        "messages":[
            HumanMessage(
                content="""
Using the PDF notes,
explain how to find the ideal
value of K in KNN.
"""
            )
        ]
    }
)

print(result["messages"][-1].content)

The agent automatically decides that it needs the RAG tool, retrieves the relevant sections from the PDF, and generates a grounded explanation.

---

Visualizing Execution with LangSmith #

This transparency helps debug tool usage, verify retrieved context, and optimize agent performance.

---

Multi-Utility Agent Architecture #

Because RAG is implemented as a tool, it integrates seamlessly with other capabilities.

The same chatbot can:

• answer general knowledge questions
• search uploaded PDFs
• solve mathematical calculations
• query external APIs
• access MCP tools
• perform web searches

without changing the overall architecture.

---

Advantages of This Approach #

• Modular and extensible design
• Reduced hallucinations through grounded retrieval
• Secure access to private documents
• Efficient semantic search using embeddings
• Intelligent tool selection with LangGraph
• Easy integration with additional tools such as MCP servers, SQL databases, search engines, and calculators
• Transparent execution tracing with LangSmith

---

Conclusion #

Retrieval-Augmented Generation is one of the most practical techniques for building reliable AI applications. By combining semantic search with a Large Language Model, we can create assistants that answer questions using private, domain-specific knowledge instead of relying solely on pretrained data.

LangGraph takes this idea a step further by treating RAG as a tool within an agentic workflow. Rather than hard-coding retrieval into every request, the agent decides when document search is necessary and when another capability—such as a calculator, web search, or external API—is more appropriate.

This modular architecture makes the chatbot significantly more powerful, scalable, and maintainable. As your application grows, you can continue adding new tools without redesigning the core workflow, enabling a single AI assistant to handle general conversations, document analysis, real-time data retrieval, and specialized business tasks within the same interaction.

Quiz #