"A model in a notebook helps no one. Deployment is turning a trained model into a live service that answers real requests."
Level: Intermediate · Time: ~4 days · Prerequisites: Modules 11–12
Learning Objectives
By the end of this module, you will be able to:
- Explain what it means to "deploy" a model
- Compare batch and real-time serving
- Describe serving a model behind an API
- Understand why deployment is harder than it looks
1. From Trained Model to Live Service
Training produces a model file. Deployment makes that file usable by an app or user — sending it new data and getting predictions back. This is where MLOps meets data engineering: the model becomes a running, reliable service.
Concept: Most of a model's value is created after training — when it's deployed and making predictions on real data, day after day. A brilliant model that never ships delivers zero value.
2. Two Ways to Serve
| Style | How it works | Good for |
|---|---|---|
| Batch | Predict for many rows on a schedule | Nightly recommendations, risk scores |
| Real-time | Predict per request, instantly | Fraud checks, chatbots, live pricing |
Explain like I'm new: Batch prediction is like developing a whole roll of film at once, overnight. Real-time is an instant photo — you need the result now. Choose based on whether users need answers immediately or can wait for a scheduled run.
3. Serving Behind an API
The common pattern for real-time is to wrap the model in a small web service exposing an endpoint: an app sends input, the service runs the model, and returns the prediction as JSON.
POST /predict { "text": "Great product!" }
→ { "sentiment": "positive", "confidence": 0.97 }Package that service in a container (Module 11) and it runs identically anywhere — laptop to cloud.
The flow inside that endpoint is worth picturing: the request arrives, the service applies the same preprocessing used in training (tokenizing text, scaling numbers), feeds the result to the loaded model, formats the prediction as JSON, and replies — ideally in a few dozen milliseconds. Any client, in any language, can call it over plain HTTP without knowing anything about the model inside.
Try this: Think of a model as a function: input → output. Deployment just makes that function callable over the network, so any app can use it. Everything else (scaling, monitoring) is making that call reliable.
4. Why It's Harder Than It Looks
Deployment introduces new worries training never had:
- Latency — real-time predictions must be fast enough.
- Scale — handle spikes from 10 to 10,000 requests/second.
- Preprocessing parity — inputs must be transformed exactly as during training.
- Versioning — track which model version is live so you can roll back.
Common mistake: Preprocessing data differently at serving time than during training ("training/serving skew"). The model then sees inputs unlike what it learned on, and accuracy silently drops. Reuse the same transformation code in both places. A classic trap: the training pipeline scales an "age" field using statistics from the training set, but the serving code forgets to apply the same scaling — so the model receives raw ages it never learned on and quietly makes worse predictions, with no error to alert anyone.
5. Deployment Is a Beginning, Not an End
Shipping a model starts the clock on operations: is it still fast? still accurate? The world changes, and models decay — which is why monitoring (Module 15) is essential. Deployment and monitoring are two halves of running ML in production.
Concept: "Deployed" doesn't mean "done." A live model needs the same care as any production service — health checks, logging, versioning, and a plan to update it safely.
A safe update plan matters more than beginners expect. Swapping a live model for a new version all at once is risky — if the new one misbehaves, everyone feels it instantly. Teams often roll out gradually (sending a small slice of traffic to the new model first) and keep the previous version ready, so a bad release can be rolled back in seconds rather than becoming an outage.
✅ Checkpoint
- What does deploying a model actually accomplish?
- When would you choose real-time serving over batch?
- What is training/serving skew?
Answers: 1) It turns a trained model file into a live service that produces predictions on new data for apps or users. 2) When predictions are needed instantly, like fraud detection or a chatbot. 3) When data is preprocessed differently at serving time than during training, causing the model to see mismatched inputs and lose accuracy.
Key Takeaway: Deployment turns a trained model file into a live, usable service. Serve batch (scheduled bulk predictions) or real-time (instant, per request, often behind an API in a container). It's harder than training because of latency, scale, preprocessing parity, and versioning — and it's only the start: a deployed model must then be monitored and maintained.
Further Learning
Part of "MLOps & Data Engineering." Original content for this learning platform.