In the evolving landscape of Generative AI, the focus is shifting from how humans interact with Large Language Models (LLMs) to how LLMs can communicate with other machines, databases, and APIs. This guide, based on the CampusX series, explores Structured Output, a critical technique for building advanced AI systems and agents.
1. Understanding Structured Output #
Unstructured Output is the standard textual response an LLM provides (e.g., “The capital of India is New Delhi”). While humans understand this, programs find it difficult to parse reliably.
Structured Output refers to responses returned in a well-defined data format, typically JSON. For example, instead of a paragraph about a travel itinerary, the model returns a list of dictionaries with specific keys like “time” and “activity”. This makes the output easy to process programmatically and integrate into other software systems.
Why is this Important?
The sources highlight three main use cases:
- Data Extraction: Pulling specific info (e.g., candidate name, last company, scores) from unstructured resumes to save directly into a database.
- Building APIs: Converting raw product reviews into structured insights such as topics, pros, cons, and sentiment.
- Building AI Agents: Agents use “tools” (like a calculator) that require specific inputs. You cannot send the text “find the square root of 2” to a calculator; you must extract the number “2” and the operation “square root” into a structured format first.
2. The with_structured_output Function #
LangChain provides a simplified way to generate structured data for models that natively support it (like OpenAI’s GPT series).
The Workflow:
- Define a Schema (the blueprint of your data).
- Bind the schema to the model using
model.with_structured_output(Schema). - Invoke the “structured model”.
Behind the Scenes: When you call this function, LangChain generates a hidden system prompt that instructs the LLM to act as an extractor and return data strictly in JSON format.
3. Implementation Methods with Code Examples #
Method 1: TypedDict (Python Native)
TypedDict allows you to define a dictionary structure with specific keys and types.
- Pros: Native Python, provides type hints in code editors.
- Cons: No runtime validation—if the LLM returns a string instead of an integer, the code will not stop it.
from typing import TypedDict, Annotated
from langchain_openai import ChatOpenAI
# 1. Define the Schema
class Review(TypedDict):
# Annotated allows adding descriptions to guide the LLM
summary: Annotated[str, "A brief summary of the review"]
sentiment: Annotated[str, "The sentiment: Positive, Negative, or Neutral"]
# 2. Setup Model
llm = ChatOpenAI(model="gpt-4o")
structured_llm = llm.with_structured_output(Review)
# 3. Invoke
result = structured_llm.invoke("The battery life is amazing but the screen is dim.")
print(result['summary'])
Method 2: Pydantic (Recommended)
Pydantic is a powerful data validation library and is considered the “go-to” format for LangChain projects.
- Pros: Performs runtime validation, supports default values, and allows Type Coercion (e.g., automatically converting the string “32” to an integer 32).
- Field Function: You can add constraints (like a CGPA must be between 0 and 10) and detailed descriptions to help the LLM.
from pydantic import BaseModel, Field
from typing import Optional, List
# 1. Define a robust schema
class Analysis(BaseModel):
key_themes: List[str] = Field(description="Main topics discussed")
rating: int = Field(ge=1, le=5, description="Rating from 1 to 5")
reviewer: Optional[str] = Field(default=None, description="Name if available")
# 2. Bind and Invoke
structured_llm = llm.with_structured_output(Analysis)
result = structured_llm.invoke("sanjit gave this 5 stars! The processor is fast.")
# Result is a Pydantic object, accessed via dot notation
print(result.rating)
Method 3: JSON Schema (Universal)
JSON Schema is ideal for projects involving multiple programming languages (e.g., Python backend and JavaScript frontend) because JSON is a universal data format.
json_schema = {
"title": "Review",
"type": "object",
"properties": {
"summary": {"type": "string", "description": "Short summary"},
"sentiment": {"type": "string", "enum": ["Positive", "Negative"]}
},
"required": ["summary", "sentiment"]
}
structured_llm = llm.with_structured_output(json_schema)
4. Advanced Configurations and Compatibility
Choosing a Method
You can specify how the output is generated using the method parameter:
- function_calling: The default and recommended method for OpenAI models.
- json_mode: Often used for models like Anthropic Claude or Google Gemini.
What if the Model Doesn’t Support It?
Not all models have native structured output capabilities. For example, the open-source TinyLlama model will throw an error if you try to use with_structured_output. In such cases, you must use Output Parsers, which manually clean and format the raw text output.
Conclusion: Comparison Table #
| Feature | TypedDict | Pydantic | JSON Schema |
|---|---|---|---|
| Data Validation | No | Yes | Yes |
| Type Conversion | No | Yes (Coercion) | No |
| Best For | Simple Python scripts | Production Apps | Multi-language projects |
Structured Output is the bridge that allows LLMs to stop being simple chatbots and start being functional engines for modern software.
Structured Output MCQ Quiz #
Q.1 In the context of Large Language Models (LLMs), what is primarily considered an 'unstructured' output?
A JSON object with key-value pairs.
A natural language text response.
A CSV file containing tabular data.
A SQL query generated for a database.
Explanation
By default, LLMs generate natural language text, which is considered unstructured because it does not follow a predefined machine-readable format like JSON.
Q.2 What is the most significant advantage of generating structured output from an LLM?
It reduces the token usage and cost of the model.
It allows the LLM to integrate easily with other machines and systems.
It prevents the model from hallucinating facts.
It makes the LLM response more human-readable.
Explanation
Structured output enables seamless integration with APIs, databases, applications, and other software systems because the output follows a predictable format.
Q.3 When building an AI Agent that uses a calculator tool, why is structured output necessary?
Structured output increases the mathematical accuracy of the LLM.
Agents are only allowed to communicate via JSON schemas.
Calculators cannot interpret natural language instructions directly.
Tools are designed to verify the sentiment of the user prompt.
Explanation
Calculator tools expect structured inputs such as numbers and parameters. They cannot reliably understand free-form natural language responses from an LLM.
Q.4 Which LangChain function is used to simplify the process of obtaining structured data from models that natively support it?
format_as_pydantic()
invoke_as_json()
parse_output_structure()
with_structured_output()
Explanation
LangChain’s with_structured_output() method automatically configures supported models to return data matching a schema such as Pydantic, TypedDict, or JSON Schema.
Q.5 What is a major limitation of using Python's 'TypedDict' for defining output schemas in LangChain?
It requires a separate license for commercial use.
It cannot be converted into a JSON format.
It does not support string-based keys.
It does not perform runtime data validation.
Explanation
TypedDict provides static type hints for developers but performs no runtime validation, so invalid data types may pass through unnoticed.
Q.6 In Pydantic, which class must your schema inherit from to define a structured data model?
SchemaObject
BaseModel
DataClass
StructuredDict
Explanation
Every Pydantic schema inherits from BaseModel, which provides validation, parsing, serialization, and other useful features.
Q.7 What happens in Pydantic if you pass the string '25' to a field defined as an integer?
The model ignores the input and sets the value to 'None'.
Pydantic throws an Immediate ValidationError.
Pydantic automatically converts (coerces) the string to the integer 25.
The value is stored as a string regardless of the definition.
Explanation
Pydantic performs type coercion whenever possible. Since ’25’ is a valid numeric string, it is automatically converted into the integer 25.
Q.8 Which scenario would most benefit from using 'JSON Schema' over 'Pydantic' for defining structured output?
When running models that do not support JSON mode.
A project built entirely in Python.
A project where the schema needs to be shared between Python and JavaScript systems.
When the developer wants the simplest possible syntax.
Explanation
JSON Schema is language-independent, making it ideal for projects where multiple programming languages need to use the same schema definition.
Q.9 What is the purpose of the 'Field' function in Pydantic schemas?
To increase the speed of the LLM's response generation.
To define metadata like descriptions and constraints for specific attributes.
To encrypt the data before sending it to the LLM.
To tell the LLM which database table to use.
Explanation
Field() allows developers to add descriptions, validation rules, default values, and constraints that help both Pydantic and the LLM understand the expected data.
Q.10 If you are using an open-source model like 'TinyLlama' that does not natively support structured output, what is the recommended approach in LangChain?
Use Pydantic with the 'with_structured_output' function.
Convert the TinyLlama model into a GPT-4 model.
Use Output Parsers to process the raw text output.
Force the model to use 'function_calling' mode.
Explanation
For models without native structured output support, LangChain recommends using Output Parsers to transform raw text responses into structured data.
Q.11 In the 'with_structured_output' function, when should you typically set the 'method' parameter to 'function_calling'?
When you want the output in a raw string format.
When you are using a model that only supports JSON mode.
When the model needs to call a human for help.
When using OpenAI models that support specialized API calls for tools.
Explanation
The function_calling method leverages OpenAI’s tool-calling capability to produce reliable structured outputs and is the recommended approach for supported models.
Q.12 How can you make a field optional in a Pydantic model so that it doesn't cause an error if the LLM fails to provide it?
Set the type to 'NoneType'.
Wrap the type in 'Optional' and provide a default value of 'None'.
Delete the field from the class definition.
Use the 'skip_validation' decorator.
Explanation
(or type | None in newer Python versions) with a default value of None, allowing the field to be omitted without raising a validation error.”]
