Module 06: RAG Fundamentals

Duration: 5 hours | Difficulty: Intermediate | Prerequisites: Modules 01–05

Learning Objectives

By the end of this module, you will be able to:

Explain the RAG architecture and why it solves LLM knowledge limitations
Load documents from various formats (PDF, web, Markdown, CSV)
Implement multiple chunking strategies and select the right one
Build a complete end-to-end RAG pipeline from scratch
Evaluate RAG system quality with quantitative metrics

Models forget your private wiki, today’s prices, and post-cutoff news. RAG (retrieval-augmented generation) fetches relevant chunks at query time and pastes them into the prompt so the model grounds on your data.

Analogy: Open-book exam vs closed-book. RAG is the open book—still your brain writes the answer, but the pages are real.

┌─────────────────────────────────────────────────────────┐
│                     RAG Pipeline                         │
│                                                          │
│  User Query ──▶ Retrieve ──▶ Augment ──▶ Generate       │
│       │           │            │            │            │
│  "What is X?"   Search      Inject       LLM produces   │
│                vector DB    context      grounded answer │
│                for top-k    into prompt                  │
│                matches                                    │
└─────────────────────────────────────────────────────────┘

Without RAG	With RAG
Model guesses from weights	Model cites retrieved text
Stale by default	Update index when docs change
Risky on proprietary facts	Still need guardrails, but better

Key Takeaway

RAG moves knowledge from weights to documents you control.

2. Document Loading

Formats you’ll see in the wild

Format	Loader idea	Gotchas
Plain text / MD	`read_text`	Encoding, huge files
PDF	Page-by-page extract	Tables break; OCR sometimes
CSV	Row → mini-doc	Which columns become “content”?
HTML / web	Parse main body	Boilerplate, nav noise

Plain-English pipeline: normalize encoding → strip boilerplate → attach metadata (source, page, url, ingested_at).

Try This! Load the same PDF with two libraries; diff the extracted text for one table-heavy page.

3. Chunking Strategies

Chunking is where RAG wins or dies: bad splits = missing context or polluted retrieval.

Fixed-Size Chunking

Sliding window with overlap—dead simple, sometimes cuts mid-thought.

Recursive Character Splitting

Try \n\n, then \n, then sentence boundaries—keeps paragraphs together when possible.

Semantic Chunking

Cluster sentences by embedding similarity—slower, often smoother chunks.

Choosing a Chunking Strategy

Strategy	Best for	Pros	Cons
Fixed-size	Uniform blobs	Predictable	Mid-sentence tears
Recursive	Docs with headings	Structure-aware	Needs tuning
Semantic	Messy mixed content	Coherent chunks	Cost + complexity

Concept: Overlap between chunks reduces “answer was on the seam” failures.

Fun Fact: Teams often spend more time on chunking than on embedding model choice.

4. End-to-End RAG Pipeline

Key Example: Minimal ingest → retrieve → generate loop: embed with a local model, store in Chroma, answer with OpenAI using injected context.

python

[object Object], chromadb
,[object Object], openai ,[object Object], OpenAI
,[object Object], sentence_transformers ,[object Object], SentenceTransformer


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],[,[object Object],]:
    chunks, i = [], ,[object Object],
    ,[object Object], i < ,[object Object],(text):
        chunks.append(text[i : i + size])
        i += size - overlap
    ,[object Object], chunks


,[object Object], ,[object Object],:
    ,[object Object], ,[object Object],(,[object Object],):
        ,[object Object],.llm = OpenAI(api_key=,[object Object],)
        ,[object Object],.emb = SentenceTransformer(,[object Object],)
        ,[object Object],.db = chromadb.Client()
        ,[object Object],.col = ,[object Object],.db.create_collection(,[object Object],, metadata={,[object Object],: ,[object Object],})

    ,[object Object], ,[object Object],(,[object Object],):
        texts, metas, ids = [], [], []
        ,[object Object], doc ,[object Object], docs:
            ,[object Object], j, ch ,[object Object], ,[object Object],(chunk_text(doc[,[object Object],])):
                texts.append(ch)
                metas.append({,[object Object],: doc[,[object Object],], ,[object Object],: j})
                ids.append(,[object Object],)
        vectors = ,[object Object],.emb.encode(texts).tolist()
        ,[object Object],.col.add(documents=texts, metadatas=metas, embeddings=vectors, ids=ids)

    ,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],:
        qv = ,[object Object],.emb.encode(question).tolist()
        res = ,[object Object],.col.query(query_embeddings=[qv], n_results=k)
        ctx = ,[object Object],.join(res[,[object Object],][,[object Object],])
        r = ,[object Object],.llm.chat.completions.create(
            model=,[object Object],,
            messages=[
                {,[object Object],: ,[object Object],, ,[object Object],: ,[object Object],},
                {,[object Object],: ,[object Object],, ,[object Object],: ,[object Object],},
            ],
            temperature=,[object Object],,
        )
        ,[object Object], r.choices[,[object Object],].message.content


rag = MiniRAG()
rag.ingest(
    [
        {,[object Object],: ,[object Object],, ,[object Object],: ,[object Object],},
        {,[object Object],: ,[object Object],, ,[object Object],: ,[object Object],},
    ]
)
,[object Object],(rag.ask(,[object Object],))

5. RAG Evaluation

You’re grading two systems: retrieval + generation.

Signal	What it catches
Hit rate @k	Did the right chunk appear in top-k?
Faithfulness	Answer supported by retrieved text?
Answer relevance	Actually addresses user question?
Latency / cost	Prod viability

LLM-as-judge (carefully): prompt a model to score faithfulness 1–5 with the context attached—cheap triage, not courtroom evidence.

Human process	Automation helper
Spot-check 20 queries	Log prompts + sources + answers
Label “good/bad”	Train a lightweight classifier later

Try This! For one failure, ask: retrieval miss or generation ignored context? Fix the right layer.

Practice Exercises

Exercise	Focus
Loader bake-off	PDF vs MD vs HTML cleanup
Chunk grid search	Overlap + size vs answer quality
Golden set	30 questions with “must include” chunk IDs
Safety	Prompt injection via uploaded doc text

Mini-Project: Documentation Q&A Bot

Index your own repo’s README + docs/ → Slack or CLI bot → always show source filenames beside answers.

Key Takeaways