"An LLM only knows what it learned in training. RAG hands the agent the right facts — your facts — right when it needs them."
Learning Objectives
By the end of this module, you will be able to:
- Explain what RAG means in plain language
- Say why agents need external knowledge
- Follow the basic RAG flow
- Know when RAG helps and when it doesn't
1. What Is RAG?
RAG stands for Retrieval-Augmented Generation. In plain words: find the right information first, then let the agent answer using it. It connects an agent to knowledge it never learned in training — your notes, your company's docs, today's data.
Key idea: RAG is like giving the agent an open-book exam. Instead of answering from memory (and risking a confident mistake), it looks up the relevant page and answers from it. That's the single most effective way to make agents accurate on private or current information.
Explain like I'm new: Without RAG, asking an agent about your company's refund policy is like asking a smart stranger who has never seen your rulebook — they'll guess, and sound sure doing it. With RAG, you hand them the rulebook first and say "answer from this." Same brain, but now grounded in the real document instead of a plausible guess.
2. Why Agents Need External Knowledge
An LLM's built-in knowledge is frozen at its training cutoff and knows nothing private. RAG fixes both problems: it lets an agent answer questions about your documents and about recent events — without expensive retraining.
This matters because retraining a model on your data is slow, costly, and has to be redone every time a document changes. RAG sidesteps all of that: update the document, and the next answer reflects the change instantly. Your knowledge stays current without ever touching the model itself.
3. The Basic Flow
RAG follows four simple steps:
- Documents — you collect the text the agent should know (PDFs, notes, a wiki).
- Search / embeddings — the text is indexed so the system can find passages by meaning, not just exact words.
- Retrieved context — for each question, the most relevant passages are pulled out.
- Answer / action — those passages are added to the prompt, and the agent responds from them.
The clever part is step 2. By turning text into embeddings (numbers that capture meaning), the system can match a question like "how do I get my money back?" to a document titled "Refund Procedure" — even though they share almost no words. It searches by idea, not by keyword.
Real-world use case: A support team points a RAG agent at their help-center articles. A customer asks, "Can I return a gift without a receipt?" The agent retrieves the exact returns article, then answers using its wording — accurate, on-policy, and with no made-up rules. Update the article next week and the agent's answers update with it.
4. When RAG Helps (and When It Doesn't)
A big reason RAG is so valued is that it reduces hallucination — the tendency of an LLM to state something false with total confidence. When a model answers from its own memory, it may fill gaps with plausible-sounding guesses. When it answers from a retrieved passage sitting right in front of it, it has real text to lean on, and it can even quote or cite the source. That's a huge win for trust: an answer you can trace back to a document is one you can verify, unlike an answer that came from nowhere.
RAG isn't magic, though. It's only as good as the documents behind it — feed it an outdated handbook and it will confidently give outdated answers. It also depends on retrieval actually finding the right passage; if the search misses, the agent answers with the wrong context or none at all. So RAG shifts the hard problem from "does the model know this?" to "did we give it the right material to read?" — a problem you can actually fix by curating good documents.
Beginner tip: Reach for RAG when the answer lives in a specific body of text — a handbook, a docs site, a set of policies. It's not needed for general knowledge the model already has ("explain photosynthesis"), and it won't fix a task that really needs a tool or action (like sending an email). Match the technique to the need.
Key Takeaway: RAG (Retrieval-Augmented Generation) gives agents an open-book exam: find relevant passages, then answer from them. The flow is documents → search/embeddings → retrieved context → grounded answer. Use it to ground agents in private or up-to-date knowledge — but not as a substitute for tools or for general facts the model already knows.