Module 16: Capstone Project Guide

Duration: 8 hours (planning + kickoff; full build spans your capstone window) | Difficulty: Advanced | Prerequisites: Modules 1–15, comfortable with Python, APIs, and basic Docker

Learning Objectives

By the end of this module, you will be able to:

1. Choose among three capstone tracks (production RAG, agentic system, or fine-tuned model) based on portfolio goals and constraints.
2. Design a production-grade architecture for your chosen track, including data flow, evaluation, and safety boundaries.
3. Build a complete AI application that composes skills from this track: embeddings, retrieval, agents, evaluation, guardrails, and/or training.
4. Ship deployment-oriented artifacts: API surface, tests, monitoring hooks, and reproducible setup.
5. Self-assess using an expanded rubric with sub-criteria and peer-review prompts.
6. Anticipate common integration failures (latency, eval leakage, tool misuse) and document mitigations.
7. Present your work with architecture diagrams, metrics, and honest limitations.
8. Cross-reference the canonical spec in projects/capstone.md for deadlines and submission format.

1. Capstone Overview

You synthesize the Generative AI engineering track by shipping one primary path. Every path must include evaluation, basic safety, and documentation.

Default (Path A): production RAG assistant over your corpus with citations, streaming API, and automated quality checks.

Details, deadlines, and submission bundles live in projects/capstone.md.

2. Project Path Options (Detailed)

Path A — Production RAG system (default)

Goal: Grounded Q&A over a corpus you control.

Key Example: Persistent Chroma collection—swap in your chunks and metadata.

[object Object], chromadb ,[object Object], PersistentClient

client = PersistentClient(path=,[object Object],)
col = client.get_or_create_collection(,[object Object],)
col.add(
    ids=[,[object Object],, ,[object Object],],
    documents=[,[object Object],, ,[object Object],],
    metadatas=[{,[object Object],: ,[object Object],}, {,[object Object],: ,[object Object],}],
)
,[object Object],(,[object Object],, col.count())

Path B — AI agent with tools

Goal: Plan → act → reflect with ≥3 tools, explicit budgets, and audit logs.

Sketch: TOOLS list + naive_plan() that picks calculator vs kb_search—replace with real LLM routing in your build.

Path C — Fine-tuned or instruction-tuned model

Goal: Measurable lift on a narrow task using SFT / LoRA—not pretraining.

Dataset row shape: instruction / input / output in JSONL—tokenize + LoRA per Hugging Face PEFT docs.

3. Project Phases (All Paths)

Phase 1: Foundation (~2 hours)

Repo layout, .env.example, pins, one happy-path smoke test.

Phase 2: Intelligence (~2 hours)

Path A: better retrieval; Path B: agent loop + tool errors; Path C: first training + eval delta.

Phase 3: Quality (~2 hours)

Eval harness, guardrails, optional cache if latency hurts.

Phase 4: Production (~2 hours)

Streaming API, structured logs, Docker, architecture diagram.

4. Milestone Table (Weekly Deliverables)

Tune weeks to your academic calendar—keep one concrete artifact per week.

5. Technical Requirements

Pin versions in requirements.txt or pyproject.toml—“works on my laptop” isn’t a deployment strategy.

6. Architecture Reference (Path A — RAG + Agent)

┌──────────────────────────────────────────────────────────┐
│                    FastAPI Service                         │
│  /chat (stream) │ /ingest │ /evaluate                      │
│         ▼                                                  │
│  Guardrails: input → PII/toxic → policy                    │
│         ▼                                                  │
│  Orchestrator: RAG + tools + memory                        │
│         ▼                                                  │
│  Retrieval: rewrite? hybrid? rerank? → vector store        │
│         ▼                                                  │
│  Monitoring: cost, latency, quality, alerts                │
└──────────────────────────────────────────────────────────┘

7. Getting Started — Project Scaffold

Suggested tree (adapt freely):

8. Evaluation Rubric (Expanded with Sub-Criteria)

Path C swap: replace RAG row emphasis with data hygiene + baseline gap + no leakage; confirm weight tweaks with your instructor.

9. Practice Exercises

1. One-page memo: pick Path A/B/C with two risks + mitigations each.
2. Draft 20 eval items tagged easy / medium / adversarial.
3. Latency budget math: retrieval + LLM + cache opportunities.
4. Injection case study via poisoned doc—list two defenses.
5. Mermaid diagram peer review.

10. Common Pitfalls

11. Submission Checklist

• Path stated in README
• Core loop works (RAG or agent or tuned model)
• Guardrails proportionate to risk
• Eval pipeline with 5+ checks/metrics
• Streaming API where applicable
• Cache or documented why not
• Cost/latency logging
• Docker instructions
• 10+ automated tests
• Architecture diagram

12. Mini-Project (Module-Level)

Vertical slice (4–6 hours): ingest five docs or wire two tools or one short training run on ~100 rows.

Deliver: branch mini-capstone, scripts/demo_slice.py, stdout table + “Known limitations” (≥3 bullets).

13. Key Takeaways

• Discipline is shared: contracts, eval, ops—not just model picks.
• README + diagram are graded artifacts; draft them in week one.
• Most failures are leakage, latency, observability—plan for them explicitly.

14. Resources

• projects/capstone.md
• LangChain templates
• FastAPI tutorial
• RAGAS
• Hugging Face PEFT
• OWASP LLM Top 10

← Previous: Production Deployment