Duration: 5 hours | Difficulty: Advanced | Prerequisites: Modules 01–03, basic ML literacy
Learning Objectives
By the end of this module, you will be able to:
- Decide when fine-tuning beats prompting, RAG, or tool use
- Prepare high-quality supervised datasets for instruction tuning
- Run a cloud fine-tuning job (OpenAI-style) with proper evaluation
- Explain LoRA / QLoRA and when parameter-efficient tuning wins
- Evaluate fine-tuned models without fooling yourself with train-set leakage
1. When to Fine-Tune
| Situation | Usually try first |
|---|---|
| Must match brand voice | Prompt + style guide |
| Need fresh facts | RAG |
| Deterministic tools | Function calling |
| Thousands of labeled examples of behavior | Fine-tune |
Fine-tuning shines when you repeatedly burn tokens teaching the same quirks (format, taxonomy, tone) that a smaller tuned model could internalize.
Fun Fact: Fine-tuning teaches behavior, not a substitute for up-to-date knowledge—you still RAG for moving facts.
2. Dataset Preparation
| Rule of thumb | Why |
|---|---|
| Thousands of (input, output) pairs | Small data overfits fast |
| Hold out a true test set | Memorization feels like genius |
| Deduplicate near-copies | Leakage inflates metrics |
| License clarity | You’ll sign attestations on some APIs |
Row shape (instruction tuning): instruction, optional input, output—JSONL lines, one example per line.
| Check before you train | Pass? |
|---|---|
| Every row reviewed or sampled for toxicity/PII | ☐ |
| Train/val/test split by document or user, not random rows | ☐ |
| Output format matches what prod will send | ☐ |
| Augmentation documented (so you don’t double-count) | ☐ |
Try This! Hand-label 20 “gold” rows before you generate thousands synthetically—quality anchor beats quantity mush.
Fun Fact: Teams often discover half their “unique” examples are near-duplicates once they embed the inputs—dedupe early.
3. OpenAI Fine-Tuning
Cloud APIs accept JSONL uploads, launch jobs, return fine-tuned model IDs. You still evaluate on held-out prompts.
Key Example: Two JSONL lines showing the chat training format many APIs expect—extend to hundreds+ for real jobs.
[object Object],[object Object],[object Object], ,[object Object],[object Object],[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object],[object Object], ,[object Object],[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object],[object Object], ,[object Object],[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object],[object Object],[object Object],
,[object Object],[object Object],[object Object], ,[object Object],[object Object],[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object],[object Object], ,[object Object],[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object],[object Object], ,[object Object],[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object], ,[object Object],[object Object],[object Object],[object Object],After training: run the same eval harness on base vs ft: model with identical temperature—compare cost, latency, and error taxonomy.
4. LoRA and QLoRA
Full fine-tune updates all weights—expensive. LoRA trains small adapter matrices; QLoRA loads a quantized base in low precision to save VRAM.
| Approach | When |
|---|---|
| Full FT | Big budget + you own stack |
| LoRA | Consumer GPUs, rapid iteration |
| QLoRA | Tight VRAM, research-y setups |
| Question | LoRA angle |
|---|---|
| How many trainable params? | Often ≪1% of base—fast to ship |
| Serving | Merge adapters or load sidecar—pick one strategy |
| Multi-task | Multiple LoRA heads per base sometimes |
Concept: LoRA is surgical—you’re not rewriting the whole brain, just nudging behavior.
Key Takeaway
If you can’t explain which behavior changed after FT, your dataset was probably too broad.
5. Evaluation After Fine-Tuning
| Check | Pass criteria |
|---|---|
| Held-out accuracy / win rate | Beat baseline + statistical margin |
| Regression suite | No catastrophic forgetting on general tasks |
| Safety eval | No new jailbreaks on red-team set |
| Cost | Cheaper per request if you downshift model size |
Key Takeaway
If eval only uses training paraphrases, you measured memorization, not generalization.
Practice Exercises
Exercise 1: Dataset Audit (Beginner)
Find duplicates and near-duplicates in a CSV; report % removed.
Exercise 2: Synthetic vs Human Mix (Intermediate)
Blend 80/20 synthetic/human—compare fine-tune outcomes.
Exercise 3: LoRA Rank Sweep (Advanced)
Train three ranks; plot eval vs size.
Exercise 4: Catastrophic Forgetting (Advanced)
Benchmark general knowledge Qs before/after FT.
Exercise 5: Cost-Benefit Memo (Advanced)
When would FT ROI beat RAG for your product?
Mini-Project: Domain Expert Fine-Tuning
Pick a narrow classification or formatting task; curate 500+ rows; fine-tune smallest capable model; ship a one-page eval report with confusion highlights.
Key Takeaways
Key Takeaway
- Fine-tune for stable behavior, not secret knowledge injection.
- Dataset hygiene matters more than arch hype.
- Cloud FT APIs are ops-simple; open LoRA is cost-flexible.
- Always compare to strong baselines (prompt/RAG).
- Eval leakage is the silent killer—design splits ruthlessly.