"Training is how a network goes from random guesses to genuine skill — by making mistakes and correcting them, millions of times."
Level: Beginner · Time: ~14 min · Prerequisites: Module 4
Learning Objectives
By the end of this module, you will be able to:
- Explain how a network learns from its mistakes
- Understand loss, gradient descent, and backpropagation simply
- Know what epochs and learning rate mean
- Name the frameworks used to build networks
1. Learning = Reducing Error
A fresh network makes terrible predictions. Training fixes this with a feedback loop:
- Predict — the network guesses an answer
- Measure error — a loss function scores how wrong it was
- Adjust — tweak the weights to reduce that error
- Repeat — thousands or millions of times
Over many rounds, the loss shrinks and predictions improve. Crucially, the network never sees the "right answer" written into its weights — it only ever gets a score telling it how far off it was, and it uses that score to inch closer. Learning here is nothing more than error, measured and reduced, again and again.
Explain like I'm new: It's like learning darts blindfolded, with a friend saying "20 cm too high, 5 cm left" after each throw. You adjust, throw again, and slowly zero in on the bullseye. The loss is that feedback; adjusting your aim is updating the weights.
2. Gradient Descent & Backpropagation
Two ideas power the "adjust" step:
- Gradient descent: the loss is like a hilly landscape; the lowest point is the best model. The network feels which way is "downhill" and takes a small step that direction — over and over — until it reaches a valley.
- Backpropagation: the method that figures out how much each weight contributed to the error, so each can be nudged the right amount. It sends the error backward through the layers.
Key idea: You don't compute these by hand — frameworks do it automatically. But the intuition matters: training = repeatedly stepping downhill on an error landscape until predictions are good enough.
Explain like I'm new: Picture standing on a foggy hillside, wanting the lowest valley but able to see only a step ahead. You feel which way the ground slopes down and take a step. Repeat, and you reach a valley without ever seeing the whole map. Gradient descent is that patient, downhill shuffle — and backpropagation is what tells each weight which way "down" is for it.
3. Epochs & Learning Rate
- Epoch: one full pass through all the training data. Networks train for many epochs.
- Learning rate: how big each downhill step is. Too big and you overshoot the valley; too small and training crawls. Tuning it is part of the craft.
Real-world use case: Training a spam classifier: it labels a batch of emails, checks its answers against the truth (loss), and adjusts weights (backprop) — repeating over the whole dataset for several epochs until accuracy plateaus. Then it's ready to filter your inbox.
4. The Frameworks
You rarely code training math yourself. Two libraries dominate:
| Framework | Known for |
|---|---|
| PyTorch | Flexible, research-favorite, very popular |
| TensorFlow / Keras | Production-ready, beginner-friendly Keras API |
A few lines in either builds and trains a network — they handle gradients and backprop for you. This is a huge deal: the math that took researchers years to work out is now a single call like loss.backward(). Your job shifts from doing the calculus to deciding the architecture, the data, and the learning rate — the choices that actually make or break a model.
Hands-On: Try This
Try this: Sketch a simple "loss curve" — error on the vertical axis, epochs on the horizontal. Draw it starting high and dropping, then flattening. That flattening is the model "converging." Recognizing this shape helps you know when training is done (or stuck).
Common Mistakes
Common mistake: Training too long. Past a point, the network starts memorizing the training data (overfitting) and gets worse on new data. Watch the test-set score, and stop when it stops improving — a technique called early stopping.
✅ Checkpoint
- What does a loss function measure?
- In one line, what is gradient descent?
- What is an epoch?
Answers: 1) How wrong the model's predictions are. 2) Repeatedly stepping "downhill" on the error landscape to reduce loss. 3) One full pass through all the training data.
Key Takeaway: Networks learn by a loop: predict, measure error with a loss function, and adjust weights to reduce it. Gradient descent steps downhill on the error landscape; backpropagation decides how much to nudge each weight. Training runs for many epochs at a chosen learning rate, using frameworks like PyTorch or TensorFlow — and you stop before it overfits.
Further Learning
- Training models — Microsoft AI for Beginners
- Reducing loss — Google ML Crash Course
- PyTorch tutorials
Adapted from Microsoft's AI for Beginners (MIT License).