"Great ML teams don't retrain and redeploy by hand. They automate the whole journey — from new data to a fresh model in production — so it's fast, safe, and repeatable."
Level: Intermediate · Time: ~4 days · Prerequisites: Modules 4, 11, 13
Learning Objectives
By the end of this module, you will be able to:
- Explain what an ML pipeline automates
- Describe CI/CD and how it extends to ML (CI/CD/CT)
- Understand experiment tracking and model registries
- Recognize why automation reduces risk
1. The ML Pipeline
A model isn't built once — it's rebuilt as data grows and the world shifts. An ML pipeline automates the repeatable path: ingest data → validate it → engineer features → train → evaluate → deploy. Run it on a button or a schedule and you get a fresh, tested model without manual toil.
Concept: Automating the pipeline makes ML reproducible — the same steps produce the same result — and lets you retrain confidently whenever data changes, instead of dreading a fragile manual process.
2. CI/CD, Meet ML
In software, CI/CD automates testing and shipping code:
- CI (Continuous Integration) — every change is automatically tested.
- CD (Continuous Delivery) — passing changes are automatically deployed.
ML adds a third: CT (Continuous Training) — automatically retraining models on new data.
Explain like I'm new: CI/CD is a factory assembly line with quality gates. Code (or a model) moves along, gets tested at each station, and only ships if it passes. Nothing reaches customers untested — automatically.
3. What's Different About ML
ML pipelines test more than code — they must also check data and the model:
| Also validate… | Example check |
|---|---|
| Data | Schema and quality gates (Module 10) |
| Model quality | New model beats the current one on a test set |
| No regressions | Accuracy didn't drop on key segments |
Common mistake: Auto-deploying a newly trained model without checking it's actually better. Retraining can produce a worse model (bad new data, a bug). Always gate deployment on the new model beating the current one.
Real-world use case: A retailer's nightly pipeline retrains its demand-forecasting model. One night, a data feed arrives half-empty due to an upstream outage. The pipeline dutifully trains on the thin data and produces a worse model — but the evaluation gate compares it to the current production model, sees the accuracy dropped, and refuses to deploy it, alerting the team instead. The old model keeps serving, and a potential bad-forecast disaster becomes a routine morning fix.
4. Tracking Experiments & Models
Two tools keep ML automation sane:
- Experiment tracking (e.g., MLflow, Weights & Biases) records every training run — its data, settings, and results — so you can compare and reproduce.
- Model registry — a versioned catalog of trained models, marking which is in "staging" vs "production," enabling instant rollback.
Together these turn a vague "we retrained it a while back" into precise, reproducible facts. The registry is like a library's catalog for models: each entry records the version, when it was promoted, and how it scored, so promoting a new model or rolling back to yesterday's is a deliberate, one-step action rather than a scramble to find the right file on someone's laptop.
Try this: Imagine you deployed a model last month and results look off. Can you find exactly which data and settings produced it? With experiment tracking and a registry, yes — in seconds. Without them, it's detective work.
5. Why Automate
Automation isn't about laziness — it's about safety and speed. Manual steps invite human error at 2 a.m.; automated pipelines with quality gates catch problems before users do, and let you update models in hours instead of weeks.
Concept: The mature end state of MLOps is: new data flows in, the pipeline retrains and validates automatically, and only a proven-better model ships — with one-click rollback if something slips through. Reliability by design. Reaching that state is a journey, not a first step — most teams start by automating just the training run, then gradually add the data checks, the quality gate, and the registry as they feel each manual step's pain.
✅ Checkpoint
- What steps does an ML pipeline automate?
- What does the extra "CT" in CI/CD/CT stand for, and why does ML need it?
- Why gate deployment on the new model beating the old one?
Answers: 1) Ingest → validate → feature engineering → train → evaluate → deploy. 2) Continuous Training — automatically retraining on new data, because models decay as the world changes. 3) Retraining can produce a worse model; the gate prevents shipping a regression.
Key Takeaway: An ML pipeline automates the repeatable path from data to deployed model, making retraining reproducible. CI/CD (automated testing and delivery) extends to ML as CI/CD/CT, adding continuous training — but ML must also validate data and model quality, gating deployment on the new model beating the old. Experiment tracking and a model registry make it all traceable and reversible.
Further Learning
Part of "MLOps & Data Engineering." Original content for this learning platform.