Advanced RAG Architectures AI Prompt

Design advanced RAG patterns to improve performance beyond naive retrieval-augmented generation.

Use case: {{use_case}}
Corpus characteristics: {{corpus}} (size, structure, update frequency, domain)
Performance gap: {{gap}} (precision, recall, multi-hop reasoning, conflicting information)

1. Corrective RAG (CRAG):
   - After retrieval: evaluate the relevance of retrieved chunks using a lightweight relevance classifier
   - If all chunks are low-relevance: fall back to web search or a broader retrieval strategy
   - Corrective step prevents the LLM from generating based on irrelevant context

2. Self-RAG:
   - The LLM generates special tokens deciding: whether to retrieve, whether the retrieved context is relevant, whether the generated sentence is supported
   - Requires training or prompting the model to produce these critique tokens
   - More reliable than always retrieving regardless of whether the query needs external knowledge

3. Multi-hop RAG (for complex reasoning):
   - Simple RAG retrieves once. Multi-hop retrieves iteratively:
     Step 1: retrieve for the original query
     Step 2: based on the first retrieval, formulate a follow-up query and retrieve again
   - Handles: questions requiring synthesizing information from multiple documents
   - IRCoT (Interleaved Retrieval with Chain-of-Thought): alternate retrieval and reasoning steps

4. Fusion RAG:
   - Generate multiple query reformulations from the original question
   - Retrieve for each reformulation independently
   - Fuse all retrieved chunks (deduplicate, rank, select top-k)
   - Better recall than single-query retrieval

5. GraphRAG:
   - Build a knowledge graph from the corpus (entities and relationships)
   - Retrieve from the graph (entity-centric) in addition to or instead of chunk-based retrieval
   - Effective for: queries about relationships between entities, entity-centric Q&A
   - Microsoft GraphRAG: open-source implementation with community detection

6. Long context vs RAG trade-off:
   - Very long context models (128K+ tokens) can sometimes ingest entire documents without retrieval
   - When to prefer long context: when the entire document is needed, retrieval precision is low
   - When to prefer RAG: corpus too large for any context window, cost of long-context inference is prohibitive, retrieval quality is high

Return: architecture recommendation for the specific use case, implementation plan for the chosen pattern, and evaluation approach to verify improvement over naive RAG.