Module 45: CI/CD for ML

"Automate the path from a code change to a tested, deployed model — so shipping improvements is safe, fast, and routine."

Level: Advanced · Time: ~12 min · Prerequisites: Modules 38, 40

Learning Objectives

By the end of this module, you will be able to:

Explain CI/CD in an ML context
Understand what gets automated and tested
Recognize the extra "CT" in ML
See a typical automated flow

CI/CD (Continuous Integration / Continuous Delivery) automates testing and deploying code. For ML, it also covers data and models, not just code — so a change is validated end-to-end before reaching users. In classic software, a "change" is almost always a code edit. In ML, a change can also be new data or a retrained model — and each of those can break things just as badly as a bad line of code. A model retrained on skewed data can quietly regress even though not a single line of code moved. That's why ML CI/CD has to test more than code.

Explain like I'm new: CI/CD is a robot quality-inspector on your assembly line. Every time someone changes something, the robot automatically tests it and — if it passes — ships it. No risky manual "hope it works" deploys.

2. What Gets Automated

CI (Continuous Integration): on every code change, run tests — unit, data, and model checks (Module 38).
CD (Continuous Delivery): if tests pass, automatically deploy (to staging, then production).
CT (Continuous Training): ML's addition — automatically retrain when new data arrives or the model drifts.

Key idea: ML adds CT (Continuous Training) to CI/CD. Regular software ships when code changes; ML systems also need to ship when data changes — retraining and redeploying as the world shifts (Module 46's drift).

Concept: ML tests go beyond ordinary unit tests. A data test checks incoming data for the right schema, sane value ranges, and no sudden spike in missing fields. A model test confirms the freshly trained model clears a minimum accuracy bar and hasn't regressed against the current production model on a held-out set. Only when code, data, and model all pass does a change earn its way toward users.

3. A Typical Flow

text

push code → run tests (code + data + model)
          → build → deploy to staging
          → validate on staging
          → promote to production
          → monitor  → (drift? retrain → repeat)

Gates at each step prevent a bad change from reaching users. Tools: GitHub Actions, GitLab CI, Jenkins, plus ML-aware pipelines (Module 40).

A useful safety habit at the final gate is a canary or shadow rollout. Instead of flipping 100% of traffic to the new model, you send it 5% (canary) and watch, or you run it silently alongside the old model on real traffic without using its answers (shadow). If metrics hold, you widen the rollout; if they dip, you pull back before most users ever saw the new version.

Real-world use case: A dev improves a feature-engineering function and opens a pull request. CI automatically runs data and model tests, trains on a sample, and checks accuracy didn't drop. Only then can it merge and auto-deploy to staging. A regression is caught by the robot, not by users.

4. Why It Matters

CI/CD (+CT) makes improvements safe and frequent. Teams that automate ship better models more often with fewer incidents; teams that deploy by hand ship rarely and nervously. There's a compounding effect here: when shipping is cheap and safe, engineers ship small changes often, each easy to review and easy to roll back. When shipping is scary and manual, changes pile up into big risky releases — the exact opposite of what you want. The same automation also leaves an audit trail: every deploy is tied to a specific commit, dataset, and set of test results, so when something does slip through, you can see exactly what changed and roll back just that.

Common mistake: Automating deployment without automated tests. Then CI/CD just ships bugs faster. The value is in the gates — tests that must pass before anything reaches production. Build the checks first.

✅ Checkpoint

What does CI/CD automate?
What extra step does ML add, and why?
Why are test "gates" essential?

Answers: 1) Automatically testing and deploying changes. 2) CT (Continuous Training) — because ML must also update when data changes/drifts. 3) They stop bad changes from reaching users; without them CI/CD just ships bugs faster.

Key Takeaway: CI/CD for ML automates testing and deploying — and adds CT (Continuous Training) because ML must update when data changes, not just code. A typical flow gates each step (code+data+model tests → staging → production → monitor → retrain). It makes shipping safe and frequent — but only if the automated tests/gates come first.

Further Learning

Part of "Zero to AI Engineer." Simplified from the AI Engineer curriculum.