Module 07: Advanced RAG Patterns

Duration: 5 hours | Difficulty: Advanced | Prerequisites: Modules 05–06

Learning Objectives

By the end of this module, you will be able to:

Implement query transformation techniques (HyDE, step-back prompting)
Build multi-hop RAG for complex questions requiring multiple retrieval steps
Apply re-ranking to improve retrieval precision
Design hybrid search combining keyword and semantic approaches
Implement agentic RAG, graph RAG, and self-RAG patterns

1. The Limits of Naive RAG

Naive RAG: embed the user question → top-k → answer. It breaks when:

Symptom	Typical cause
“Right topic, wrong paragraph”	Query/document vocabulary mismatch
Needs two docs to answer	Single-hop retrieval
Code names vs marketing names	Lexical gap
Long tail noise in top-k	Embedding-only recall

Advanced stack (mental model): rewrite query → retrieve wide → re-rank tight → optionally compress context → generate.

Try This! Log the top-5 chunks for 10 failed queries—patterns jump out faster than tuning k blindly.

Ask the model for a fake but plausible answer paragraph, embed that, search with it. Questions and documents often live in different “styles”; a synthetic paragraph can land closer to real docs.

HyDE win	HyDE risk
Short keyword queries	Hypothetical text invents facts used only for search—don’t show it to users raw

Step-Back Prompting

Turn “What’s the side effect of ibuprofen for adults over 65 on blood thinners?” into a broader retrieval query like “ibuprofen drug interactions bleeding risk”—fetch textbook context first, then answer specifically.

Multi-Query Expansion

LLM writes 3–5 paraphrases; you retrieve per query and union/dedupe results. Buys recall; costs more tokens and latency.

Key Takeaway

If retrieval fails, fix the query before you swap embedding models.

3. Re-Ranking

Bi-encoders (query embedding vs doc embedding) are fast but coarse. Cross-encoders score each (query, doc) pair jointly—slower, sharper.

Key Example: Score candidate strings with a small cross-encoder; keep top 3 for the LLM.

python

[object Object], sentence_transformers ,[object Object], CrossEncoder

,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],[,[object Object],[,[object Object],, ,[object Object],]]:
    model = CrossEncoder(,[object Object],)
    scores = model.predict([(query, d) ,[object Object], d ,[object Object], docs])
    ranked = ,[object Object],(,[object Object],(docs, scores), key=,[object Object], x: x[,[object Object],], reverse=,[object Object],)
    ,[object Object], ranked[:top_k]


docs = [
    ,[object Object],,
    ,[object Object],,
    ,[object Object],,
]
,[object Object], doc, score ,[object Object], rerank(,[object Object],, docs):
    ,[object Object],(,[object Object],)

Two-stage pattern: retrieve 50 with vectors → re-rank to 5 → generate.

4. Hybrid Search (Keyword + Semantic)

Dense search misses exact SKUs, error codes, and rare tokens. Hybrid blends BM25-style keyword scores with cosine similarity.

Component	Strength
Semantic	Paraphrase robustness
Keyword	Exact matches, rare tokens

Blend with weights (e.g., 0.6 * dense + 0.4 * sparse) and renormalize—tune on your eval set.

Fun Fact: Many “vector only” demos quietly add hybrid later once users type part numbers.

5. Multi-Hop RAG

Some answers need A from doc1 plus B from doc2. Loop: retrieve → ask model “enough?” → if not, generate a follow-up search → repeat with budget cap.

knob	purpose
`max_hops`	Stop runaway loops
dedupe chunks	Avoid context bloat

6. Agentic RAG

Let the model decide when to call search_kb(query) instead of always retrieving. Tool loop from Module 08 applies directly.

Plus	Minus
Saves tokens on greetings	Harder to test; needs tracing

7. Self-RAG

Add critic steps: “Do I even need retrieval?” and “Is this answer supported?” Often implemented as extra LLM calls or structured JSON flags—trade cost for calibration.

Self-RAG idea	Plain English
Retrieve flag	“I know this” vs “I should look”
Citation check	“Each sentence maps to a chunk substring?”
Abstain path	“Context insufficient” beats confident fiction

Concept: Self-RAG is governance, not a different vector database.

Graph RAG (conceptual)

When answers require relationships (“Who reports to whom in the 2024 reorg?”), flat chunks struggle. Graph RAG links entities and relations, then retrieves neighborhoods instead of only similar paragraphs. You pay ingestion complexity for multi-hop questions.

When graph helps	When to skip
Org charts, regulations with cross-refs	Simple FAQ bots
Supply chains, biology pathways	You lack entity extraction budget

Try This! List three questions your users ask that need two named entities in one answer—if that’s common, sketch a graph.

Fun Fact: “Graph RAG” is often 80% good entity extraction and 20% graph database glamor.

Practice Exercises

Exercise	Focus
HyDE A/B	Ten queries where user speaks in “Google-ish” keywords
Re-rank sweep	Measure MRR@k before/after cross-encoder
Hybrid tuning	alpha grid on your corpus
Agentic	Tool-only search vs always-on retrieval—latency story

Mini-Project: Advanced RAG Search Engine

Build: query rewriter → hybrid retrieve (50) → cross-encoder rerank (5) → answer with citations. Ship a README with failure examples.

Key Takeaways