"Basic RAG retrieves once and answers. Agentic RAG can think, search again, use tools, and check itself — retrieval with a brain."
Level: Advanced · Time: ~13 min · Prerequisites: Modules 19, 21
Learning Objectives
By the end of this module, you will be able to:
- Explain how agentic RAG extends basic RAG
- Describe the reason-retrieve-refine loop
- Recognize when it's worth the complexity
- Combine agents, tools, and retrieval
1. Beyond One-Shot Retrieval
Basic RAG (Module 21) does a single retrieve → answer. That's great for direct questions but struggles with complex ones needing multiple lookups or reasoning. Ask basic RAG "which of our two plans is cheaper and has more storage?" and one search rarely surfaces both facts at once — it retrieves whatever is closest to the whole question and often misses half the answer. Agentic RAG puts an agent (Module 19) in charge of retrieval — deciding what to search, when to search again, and whether the answer is good enough.
Explain like I'm new: Basic RAG is looking up one fact and answering. Agentic RAG is a diligent researcher: it looks something up, realizes it needs more, searches again, cross-checks, and only then writes the answer. It reasons about its own retrieval.
Concept: The key upgrade is query rewriting. A user's raw question is often a poor search query. An agentic system can reformulate it — splitting a compound question into parts, adding context from the conversation, or trying synonyms — so retrieval actually finds the right passages.
2. The Reason–Retrieve–Refine Loop
An agentic RAG system loops:
- Reason: break the question into what needs looking up
- Retrieve: search the vector store (or multiple sources)
- Evaluate: is this enough? accurate? relevant?
- Refine: rewrite the query or search again if needed
- Answer: synthesize once it has what it needs
Key idea: The agent can rewrite queries, choose among multiple data sources, and decide when to stop — capabilities basic RAG lacks. It trades simplicity for the ability to handle multi-hop, complex questions.
3. Combining Tools + Retrieval
Agentic RAG often blends retrieval with other tools: search the docs, and call a live API, and run a calculation. The agent orchestrates them toward the goal. Retrieval simply becomes one tool among several — so the agent might treat "search the knowledge base" and "query today's exchange rate" as equal options and pick whichever the current step needs. This is where the RAG and agent threads of this course converge: an agent that can also retrieve is far more capable than one that can only call APIs, and far more flexible than a fixed retrieve-then-answer chain.
Real-world use case: "Compare our Q3 refund policy with last year's and flag changes." An agentic system retrieves this year's policy, retrieves last year's, compares them, maybe calls a summarizer tool, and reports the differences — several retrievals and steps, coordinated by the agent.
4. Worth the Complexity?
Agentic RAG is powerful but slower, costlier, and harder to debug than basic RAG.
Common mistake: Jumping to agentic RAG for simple Q&A. If one retrieval answers the question, basic RAG is faster, cheaper, and more reliable. Reserve agentic RAG for genuinely complex, multi-step questions — and cap the agent's loops so it can't spiral.
Hands-On: Try This
Try this: Write a question that basic RAG would fumble because it needs two separate lookups (e.g., "Which of our two products has the better warranty, and by how long?"). Notice how it needs retrieve → retrieve → compare — exactly the loop agentic RAG provides.
✅ Checkpoint
- How does agentic RAG differ from basic RAG?
- What can the agent do that basic retrieval can't?
- When should you stick with basic RAG?
Answers: 1) An agent controls retrieval, looping and refining, vs. one-shot retrieve-and-answer. 2) Rewrite queries, use multiple sources/tools, decide when it has enough. 3) For simple questions answered by a single retrieval.
Key Takeaway: Agentic RAG puts an agent in charge of retrieval — a reason → retrieve → evaluate → refine → answer loop that can rewrite queries, use multiple sources and tools, and self-check. It handles complex, multi-hop questions basic RAG can't, but it's slower and costlier — so use it only when simple RAG falls short, with capped loops.
Further Learning
Adapted from the LangChain for Beginners curriculum (MIT License).