--- title: RAG Fusion description: "Retrieval-Augmented Generation (RAG) has transformed how organizations interact with custom knowledge bases. By connecting Large Language Models (LLMs) to external vector databases, RAG systems enable AI models to generate accurate, context-aware answers grounded in private documents. However, standard RAG systems…" url: "https://csestudy247.com/docs/rag-fusion/" updated: "2026-09-10" category: RAG --- # RAG Fusion Retrieval-Augmented Generation (RAG) has transformed how organizations interact with custom knowledge bases. By connecting Large Language Models (LLMs) to external vector databases, RAG systems enable AI models to generate accurate, context-aware answers grounded in private documents. However, standard RAG systems suffer from a major bottleneck: **query dependence**. If a user inputs an ambiguous, poorly structured, or incomplete question, the retrieval mechanism fails to locate the right document chunks. **RAG Fusion** solves this fundamental issue. By combining **LLM-driven multi-query expansion** with **Reciprocal Rank Fusion (RRF) re-ranking**, RAG Fusion dramatically elevates retrieval precision, context coverage, and answer accuracy. This comprehensive guide breaks down how RAG Fusion works, the mathematical intuition behind Reciprocal Rank Fusion, how it compares to other advanced retrieval techniques, and how to implement it step-by-step in modern AI applications. --- ## 1. Introduction In a standard RAG pipeline, the quality of the final generated answer is strictly bounded by the quality of the retrieved information. In fact, retrieval accounts for approximately **80% of the overall performance** of a RAG architecture. If the retrieval step fails to fetch relevant document chunks, the language model—regardless of its capabilities—cannot produce a factually correct answer. In a traditional **Vanilla RAG** setup, the raw input query is converted into a vector embedding and compared against stored document embeddings using similarity metrics like cosine distance. This approach assumes that the user's query uses the exact vocabulary, structure, and phrasing required to match the indexed documents. In real-world applications, end users rarely ask perfect questions. They omit context, use informal terminology, or frame questions vaguely. Relying solely on a single user query creates a fragile system. RAG Fusion eliminates this single point of failure by taking a raw query, generating multiple alternative sub-queries using an LLM, performing parallel retrieval, and re-ranking the combined results using Reciprocal Rank Fusion. --- ## 2. What Is RAG Fusion? **RAG Fusion** is an advanced Retrieval-Augmented Generation workflow that enhances standard RAG through two core mechanisms: 1. **Multi-Query Expansion**: Using an LLM to automatically rephrase and expand a single user query into multiple semantically similar sub-queries with varied phrasing and keywords. 2. **Reciprocal Rank Fusion (RRF)**: A score-agnostic re-ranking algorithm that combines multiple ranked lists of retrieved documents into a single, optimized context block based on document rank positions and multi-query consistency. graph TD A\["💬 Input User Query"\] --> B\["🧠 LLM Query Generator"\] B --> S1\["🔍 Sub-Query 1"\] B --> S2\["🔍 Sub-Query 2"\] B --> S3\["🔍 Sub-Query 3"\] S1 --> V1\["🗄️ Vector Search 1"\] S2 --> V2\["🗄️ Vector Search 2"\] S3 --> V3\["🗄️ Vector Search 3"\] V1 --> RRF\["🔀 Reciprocal Rank Fusion (RRF Re-ranking)"\] V2 --> RRF V3 --> RRF RRF --> RC\["📄 Re-Ranked Context"\] RC --> LLM\["🤖 LLM Generation Phase"\] B --> B1\["• Query expansion"\] B --> B2\["• Diverse sub-queries"\] RRF --> R1\["• Merges rank lists"\] RRF --> R2\["• No score normalization"\] style A fill:#6f42c1,color:#fff style B fill:#005cbf,color:#fff style S1 fill:#17a2b8,color:#fff style S2 fill:#17a2b8,color:#fff style S3 fill:#17a2b8,color:#fff style V1 fill:#ffc107,color:#000 style V2 fill:#ffc107,color:#000 style V3 fill:#ffc107,color:#000 style RRF fill:#28a745,color:#fff style RC fill:#dc3545,color:#fff style LLM fill:#6f42c1,color:#fff By fusing multiple search passes into one unified ranking, RAG Fusion ensures that the language model receives rich, high-conviction context even when the original user input is sub-optimal. --- ## 3. Key Concepts Understanding RAG Fusion requires familiarity with several core concepts across the RAG ecosystem: - **Vanilla RAG**: The traditional single-query pipeline where a query is converted into an embedding, searched against a vector database, and passed to an LLM. - **Query Dependence Bottleneck**: The vulnerability of standard RAG where retrieval quality relies heavily on the user asking the question with ideal keywords. - **Multi-Query Expansion**: Generating N alternate versions of a user prompt using an LLM to capture different phrasing styles, perspectives, and keyword variants. - **Reciprocal Rank Fusion (RRF)**: An ensemble re-ranking algorithm that evaluates documents across multiple search result lists, scoring them based on rank positions rather than raw distance scores. - **Document Consistency**: The presence of a candidate document across multiple sub-query search runs. RRF heavily rewards documents that appear consistently across search variations. - **Smoothing Factor (k)**: A constant in the RRF equation (typically set to 60) that balances the weight given to individual top ranks versus multi-query consistency. --- ## 4. Detailed Explanation ### The Evolution of Retrieval Architectures To understand why RAG Fusion is necessary, it helps to review how advanced retrieval strategies evolved to solve specific RAG limitations: #### 1. Contextual Compression Retriever Standard text splitters split documents based on character or token counts without recognizing semantic topic boundaries. Consequently, a single chunk often contains noisy, irrelevant topics alongside useful information. Contextual Compression uses an LLM as an **extractor model** to compress retrieved chunks, stripping away noise and preserving only the query-relevant sentences. #### 2. Self-Query Retriever When user queries contain both unstructured text and structured metadata requirements (e.g., filtering by date or category), Self-Query uses an LLM to parse the raw input into a structured JSON query. A **Translator** component converts this structured query into retriever-specific filter language for hybrid execution. #### 3. Parent Document Retriever This technique resolves the trade-off between **embedding quality** (which favors small text chunks) and **contextual volume** (which favors large text chunks). It stores large parent chunks in a key-value Document Store and splits them into smaller child chunks for vector indexing. During retrieval, similarity search identifies top child chunks, but the system fetches and injects their corresponding large parent chunks into the final context. #### 4. Multi-Query Retriever Multi-Query uses an LLM to rephrase an ambiguous prompt into multiple alternative queries. It executes vector search for each sub-query, collects all retrieved chunks, and performs simple **deduplication**. #### 5. RAG Fusion (The Missing Link) While Multi-Query improves search coverage, simple deduplication throws away valuable ranking metadata. A document ranked #1 across all sub-queries is treated identical to a document ranked #10 in only one pass. **RAG Fusion addresses this gap by adding Reciprocal Rank Fusion (RRF)**. Instead of merely removing duplicates, RAG Fusion evaluates the relative position of every chunk in every search pass, calculating a unified score that mathematically prioritizes consistent, top-performing documents. --- ## 5. How It Works The complete RAG Fusion workflow operates across **seven structured steps**: graph TD A\["💬 User Query"\] --> B\["🔀 Multi-Query Expansion"\] B --> C\["⚡ Parallel Searches"\] C --> D\["📊 RRF Scoring"\] D --> E\["🎯 Re-Ranking"\] E --> F\["📄 Context Build"\] F --> G\["🤖 LLM Generation"\] A --> A1\["Original question"\] B --> B1\["Sub-query 1"\] B --> B2\["Sub-query 2"\] B --> B3\["Sub-query 3"\] C --> C1\["Vector Search 1"\] C --> C2\["Vector Search 2"\] C --> C3\["Vector Search 3"\] D --> D1\["Merge rank lists"\] D --> D2\["Score normalization"\] E --> E1\["Cross-encoder"\] E --> E2\["Top-K selection"\] F --> F1\["Combine chunks"\] F --> F2\["Build prompt"\] G --> G1\["Grounded answer"\] style A fill:#6f42c1,color:#fff style B fill:#005cbf,color:#fff style C fill:#17a2b8,color:#fff style D fill:#ffc107,color:#000 style E fill:#28a745,color:#fff style F fill:#dc3545,color:#fff style G fill:#6f42c1,color:#fff ### Step 1: User Query Entry The user submits a prompt or question to the system (e.g., *"How does document processing and vector indexing work?"*). ### Step 2: Multi-Query Generation via LLM The system routes the input query to an LLM paired with a specialized system prompt and structured schema (such as a Pydantic `BaseModel`). The LLM generates N alternative sub-queries (typically 3 to 5) that capture the underlying intent using different sentence structures and vocabulary. ### Step 3: Parallel Multi-Query Retrieval Each generated sub-query is sent independently to the vector database or document retriever (such as similarity search, MMR, or keyword search). The retriever returns candidate document chunks for every sub-query. ### Step 4: Reciprocal Rank Fusion (RRF) Scoring When a system generates multiple sub-queries to retrieve relevant documents, it ends up with several separate ranked lists. The challenge is: **how do you combine these lists into one unified ranking?** The answer is **Reciprocal Rank Fusion (RRF)** — a technique that ignores original similarity distance scores and instead calculates a unified score based on the *rank positions* of documents across all sub-queries. --- The RRF Formula For every unique document, the system calculates a unified RRF score using the following formula: RRFScore(d∈D)=∑q∈Q1Rank(d,q)+k **Where:** | Symbol | Meaning | | --- | --- | | d | A candidate document chunk | | Q | The set of generated sub-queries | | Rank(d,q) | The 1-based index position of document d in the results for sub-query q | | k | The smoothing constant (default value is **60**) | > **Note:** If a document does not appear in the top results for a sub-query, its score contribution for that query is **0**. --- ### Step 4: Score Aggregation The system aggregates all candidate document lists. Ignoring original similarity distance scores, it calculates a unified RRF score for every unique document using the formula above. ### Step 5: Re-Ranking & Selection All unique candidate documents are sorted in **descending order** based on their cumulative RRF scores. The **top-K** documents with the highest scores are selected. ### Step 6: Context Augmentation The selected top-ranked chunks are concatenated into a clean context block alongside the original user query and inserted into a generation prompt. ### Step 7: Final Response Generation The augmented prompt is passed to the generation LLM, which synthesizes a factual, comprehensive answer grounded strictly in the provided context. --- #### Worked Numerical Example To see how RRF evaluates documents in practice, consider an example with **3 sub-queries** (Q1,Q2,Q3​) and a smoothing constant **k=60**. #### Retrieved Results across Sub-Queries | Query | Rank 1 | Rank 2 | Rank 3 | | --- | --- | --- | --- | | Q1 | Document 2 | Document 5 | Document 1 | | Q2 | Document 3 | Document 2 | Document 1 | | Q3 | Document 2 | Document 4 | Document 3 | --- #### Step-by-Step RRF Calculations #### Document 1 | Query | Rank | Calculation | Score | | --- | --- | --- | --- | | Q1 | 3 | 13+60=1633+601​=631​ | ≈ 0.01587 | | Q2 | 3 | 13+60=1633+601​=631​ | ≈ 0.01587 | | Q3 | — | Absent | 0 | **Total RRF Score for Doc 1** = 0.01587 + 0.01587 + 0 = **0.03174** --- #### Document 2 | Query | Rank | Calculation | Score | | --- | --- | --- | --- | | Q1 | 1 | 11+60=1611+601​=611​ | ≈ 0.01639 | | Q2 | 2 | 12+60=1622+601​=621​ | ≈ 0.01613 | | Q3 | 1 | 11+60=1611+601​=611​ | ≈ 0.01639 | **Total RRF Score for Doc 2** = 0.01639 + 0.01613 + 0.01639 = **0.04891** --- #### Document 3 | Query | Rank | Calculation | Score | | --- | --- | --- | --- | | Q1 | — | Absent | 0 | | Q2 | 1 | 11+60=1611+601​=611​ | ≈ 0.01639 | | Q3 | 3 | 13+60=1633+601​=631​ | ≈ 0.01587 | **Total RRF Score for Doc 3** = 0 + 0.01639 + 0.01587 = **0.03226** --- ### Final Re-Ranked Results | Rank | Document | RRF Score | | --- | --- | --- | | 🥇 1 | Document 2 | **0.04891** | | 🥈 2 | Document 3 | **0.03226** | | 🥉 3 | Document 1 | **0.03174** | --- ### Key Takeaways ### Consistency Wins Document 2 took the top position because it appeared in **all three query passes**. ### Rank as a Tie-Breaker Both Document 3 and Document 1 appeared in two query passes. Document 3 won second place because it achieved a **Rank 1** position in Q2, whereas Document 1 only reached Rank 3. --- ## Understanding the Role of the Smoothing Factor (k) The constant k in the denominator governs the trade-off between **top rank dominance** and **multi-query consistency**: | k Value | Behavior | Effect | | --- | --- | --- | | **Large k** (e.g., 100) | Shrinks score differences between Rank 1 (1101≈0.00991011​≈0.0099) and Rank 2 (1102≈0.00981021​≈0.0098) | Minimizes the impact of individual rank order; places almost all weight on **how consistently** a document appears across sub-queries | | **Small k** (e.g., 1) | Creates a steep drop between Rank 1 (12=0.521​=0.5) and Rank 2 (13≈0.33331​≈0.333) | Heavily favors documents that rank **#1 in a single pass**, even if they are absent elsewhere | | **Default *k*=60** | Empirically verified optimal balance | Provides strong consistency rewards while preserving relative rank weight | --- ## 7. Comparison The comparison table below details how Vanilla RAG, Multi-Query RAG, and RAG Fusion differ structurally: | Feature / Metric | Vanilla RAG | Multi-Query RAG | RAG Fusion | | --- | --- | --- | --- | | **Query Strategy** | Single raw user query | Multiple rephrased queries | Multiple rephrased queries | | **Retrieval Passes** | Single vector search | Multiple parallel searches | Multiple parallel searches | | **Result Merging** | Direct prompt insertion | Simple deduplication | **Reciprocal Rank Fusion (RRF)** | | **Rank Awareness** | None | Discards rank metadata | **Scores based on rank position** | | **Consistency Reward** | None | None | **High score boost for multi-pass presence** | | **Query Failure Risk** | High (vulnerable to user phrasing) | Low | **Extremely Low** | | **System Overhead** | 1 LLM call | 2 LLM calls + N searches | 2 LLM calls + N$searches | | **Retrieval Precision** | Moderate | Moderate / High | **Exceptional** | --- ## 8. Advantages and Limitations ### Advantages - **High Fault Tolerance**: Users do not need to phrase questions perfectly. The system automatically explores semantic variations. - **Broader Context Coverage**: Captures complementary context chunks scattered across different phrasing styles. - **Model & Score Agnostic**: Works purely on relative rank positions, eliminating dependencies on raw vector similarity metrics. - **Hybrid Search Compatibility**: Easily blends different retrieval methods (e.g., combining BM25 keyword search with Chroma vector search via `EnsembleRetriever`). ### Limitations - **Increased Latency**: Generating sub-queries and executing multiple search calls adds system response time compared to single-pass retrieval. - **Higher API & Compute Costs**: Multi-query expansion requires an extra LLM call, increasing token consumption. - **Network Data Transfer**: Retrieving multiple candidate lists increases network data movement in enterprise production settings. --- ## 9. Real-World Applications RAG Fusion is ideal for production applications where query variability is high and retrieval accuracy is mission-critical: - **Enterprise Document Search**: Internal knowledge bases where employees search complex policy documents using informal phrasing. - **Customer Support Automation**: Self-service bots where customers report technical issues using non-standard descriptions. - **Legal & Regulatory Discovery**: High-stakes legal QA platforms where missing a relevant clause due to vocabulary mismatch creates compliance risks. - **Hybrid Search Architectures**: Production engines that combine sparse keyword search (BM25) with dense vector search, using RRF to blend keyword accuracy with semantic depth. --- ## 10. Important Points for Revision - **RAG Fusion Equation**: Combines **Multi-Query Expansion** with **Reciprocal Rank Fusion (RRF)**. - **RRF Formula**: RRFScore(d∈D)=∑q∈Q1Rank(d,q)+k - **Default Constant**: The smoothing factor $k$ is empirically set to **60**. - **Core Philosophy**: Prioritizes **document consistency** across multiple sub-query passes over single-pass outlier ranks. - **Rank Over Raw Distance**: RRF discards raw similarity scores and operates exclusively on **1-based rank positions**. - **Trade-off**: Higher retrieval accuracy and context coverage in exchange for moderate increases in latency and token cost. --- ## 11. Interview / Exam Questions ### Question 1: What is the key difference between Multi-Query RAG and RAG Fusion? **Answer:** Multi-Query RAG generates multiple query variations and simple deduplicates the retrieved document lists. RAG Fusion takes Multi-Query RAG further by applying Reciprocal Rank Fusion (RRF) to score and re-rank candidate documents based on their rank position and consistency across all search passes. ### Question 2: Why does RRF use rank positions rather than raw vector similarity scores? **Answer:** Raw vector similarity scores (such as cosine distance or Euclidean distance) vary across vector spaces, distance metrics, and index types. Rank positions provide a normalized metric that allows fair, standardized score aggregation across different query passes and retriever types. ### Question 3: How does changing the constant k impact the RRF algorithm? **Answer:** The constant k controls the balance between top rank dominance and document consistency. A smaller k (k=1) increases score gaps between top ranks, favoring single-pass top hits. A larger k(k=100) flattens rank gaps, making multi-query consistency the dominant scoring factor. The value k=60 provides the ideal balance. ### Question 4: How can RAG Fusion be implemented with hybrid search? **Answer:** RAG Fusion can use `EnsembleRetriever` to combine different search algorithms—such as BM25 keyword search and dense vector similarity search. Each retriever returns ranked results, and RRF merges them into a single consolidated list. --- ## 12. Quick Revision **RAG Fusion** upgrades traditional RAG pipelines into resilient multi-query search engines. By generating diverse sub-queries with an LLM and applying **Reciprocal Rank Fusion (RRF)** to re-rank documents based on rank consistency, RAG Fusion guarantees that the most relevant context reaches the language model. For production AI applications where accuracy and coverage are critical, RAG Fusion represents one of the most effective retrieval enhancements available today. ## RAG Fusion Quiz 1. According to the session, what percentage of the overall RAG pipeline does the retrieval phase account for in terms of importance? - Around 20% - Around 50% - Around 80% - 100% It covers the majority of the pipeline's effectiveness before augmentation and generation. [Discuss it](#) Explanation The retrieval step accounts for approximately 80% of the overall RAG pipeline, making it the most critical phase to understand and optimize . 2. What is identified as the biggest bottleneck in a standard Vanilla RAG pipeline? - Inability of vector stores to save metadata - Dependency on the user's input query phrasing and word choices - Failure of document loaders to ingest PDFs - Lack of pre-trained embedding models Consider who provides the query and whether developers can control user phrasing. [Discuss it](#) Explanation The user's input query is the main bottleneck in Vanilla RAG because retrieval accuracy directly depends on whether the user asks the right question using the right words . 3. How does a Contextual Compression Retriever handle noisy or irrelevant information in retrieved text chunks? - It permanently deletes stop words from the vector database - It uses an LLM extractor model to discard non-relevant context and retain only query-relevant facts - It converts text chunks into audio vectors - It merges all documents into a single chunk without filtering Think about how an LLM can filter out irrelevant topics from a chunk containing multiple topics. [Discuss it](#) Explanation Contextual Compression Retriever passes the query and retrieved chunks to an LLM extractor model, which discards noisy context and keeps only information relevant to the query . 4. What role does the Translator component play in a Self-Query Retriever pipeline? - It translates text between foreign human languages - It converts the LLM's structured JSON query into a filter format the retriever understands - It converts vector embeddings back into plain text - It splits PDF pages into smaller chunks It acts as a bridge between the structured JSON output of the LLM and the vector store retriever. [Discuss it](#) Explanation The Translator component takes the structured query in JSON format generated by the LLM and converts it into a filter query that the specific retriever can execute . 5. In a Parent Document Retriever, how is the trade-off between retrieval quality and context volume resolved? - By generating embeddings directly for entire documents without chunking - By performing similarity search on small child chunks and retrieving larger parent chunks via parent IDs - By using single-character chunking strategies - By discarding child chunks after storing them in the vector store Small chunks offer higher embedding and retrieval quality, while parent chunks provide richer context. [Discuss it](#) Explanation Parent Document Retriever uses small child chunks stored in the vector store for accurate similarity search, then fetches the corresponding larger parent chunk from a Document Store using parent IDs . 6. What is the main goal of using a Multi-Query Retriever? - To encrypt queries before sending them to the vector store - To rephrase the input query into multiple sub-queries for greater retrieval coverage - To eliminate the need for vector databases - To summarize documents into single-sentence summaries Think about generating multiple sub-queries with different phrasing to catch different vocabulary styles. [Discuss it](#) Explanation Multi-Query Retriever uses an LLM to rephrase the input query into multiple sub-queries with different wordings to achieve greater retrieval coverage across the document base . 7. What critical step does a standard Multi-Query Retriever perform that causes it to lose ranking information? - It compresses vector embeddings - It simply deduplicates retrieved document lists without re-ranking them based on rank positions - It deletes candidate documents permanently - It converts documents to JSON format Consider what happens when search results from multiple queries are combined into one list without re-ordering. [Discuss it](#) Explanation Standard Multi-Query Retriever combines retrieved document lists and removes duplicate items, but it discards the rank position metadata of documents across search passes . 8. What are the two core steps performed in a RAG Fusion workflow? - Text Summarization and Model Fine-Tuning - LLM Multi-Query Expansion and Reciprocal Rank Fusion (RRF) Re-ranking - Document Partitioning and Data Encryption - Keyword Filtering and Speech Synthesis RAG Fusion expands the user query into sub-queries first, then applies a specific re-ranking algorithm. [Discuss it](#) Explanation RAG Fusion first uses an LLM to generate multiple sub-queries from the input prompt, then re-ranks all retrieved documents using Reciprocal Rank Fusion (RRF) . 9. What is the mathematical formula for calculating the RRF score of a document d across queries q? - RRF Score = 1 / (Rank(d) \* k) - RRF Score = Sum of 1 / (Rank(d, q) + k) across all queries q - RRF Score = Cosine Similarity(d) + k - RRF Score = Rank(d) / 60 The score sums the reciprocal of document rank plus a constant for each query pass. [Discuss it](#) Explanation The RRF score for document d is calculated by summing 1 / (Rank(d, q) + k) across all sub-query retrieval passes q in which document d appears . 10. What property in candidate documents does Reciprocal Rank Fusion (RRF) reward most heavily? - Highest cosine similarity score in a single query pass - Document consistency across multiple sub-query retrieval passes - Longest character length of the text chunk - Alphabetical order of the text Think about why a document that appears across all sub-queries receives a higher cumulative score. [Discuss it](#) Explanation RRF prioritizes document consistency, giving higher cumulative scores to documents that consistently appear across multiple sub-query search passes . 11. What is the empirically proven default value for the smoothing constant k in the RRF formula? - 1 - 10 - 60 - 100 It is the standard constant value established through experimentation for balancing rank and consistency. [Discuss it](#) Explanation Experiments show that setting the smoothing constant k to 60 provides an optimal balance between rank position and multi-query consistency . 12. What happens if you significantly increase the constant k (e.g. setting k = 100 or higher) in the RRF formula? - Differences between ranks decrease, placing almost all weight on document consistency - Top-ranked documents receive exponentially higher scores - RRF scores drop to zero for all documents - The score reverts to raw vector cosine similarity A larger denominator makes fractional differences between Rank 1 and Rank 2 much smaller. [Discuss it](#) Explanation Increasing k increases the denominator and reduces score differences between consecutive ranks, diminishing the impact of rank position and putting primary weight on consistency . 13. Why is the algorithm named 'Reciprocal Rank Fusion'? - It computes inverse matrix transformations on vector stores - It fuses search results by taking the mathematical reciprocal of document position ranks - It interchanges the roles of LLM and retriever - It multiplies embedding vectors reciprocally Look at the fraction term 1 / (Rank + k) used during merging. [Discuss it](#) Explanation The algorithm is called Reciprocal Rank Fusion because it fuses ranked lists by taking the reciprocal (1 / Rank) of each document's rank position . 14. Which of the following is cited as an operational disadvantage of using RAG Fusion? - Inability to handle PDF documents - Higher latency and increased API / network call costs - Complete loss of context coverage - Incompatibility with vector databases Generating sub-queries requires additional LLM calls and multiple retrieval steps over the network. [Discuss it](#) Explanation Disadvantages of RAG Fusion include increased latency from multiple retrieval passes and higher operational costs due to additional LLM API calls and network data transfer . 15. In the Python implementation shown in the session, how is structured output obtained from the LLM when generating sub-queries? - By parsing plain text strings using regular expressions - By defining a Pydantic model (SubQuerySchema) and calling llm.with_structured_output() - By converting SQL queries into JSON format - By manually splitting text on commas Think about defining a Pydantic schema with a list of strings field. [Discuss it](#) Explanation The implementation defines a Pydantic class SubQuerySchema inheriting from BaseModel and passes it to llm.with_structured_output() to obtain a structured list of sub-queries from the LLM .