"No textbook, no answer key — just a world, a goal, and the freedom to try. Reward what works, and behavior improves all by itself."
Learning Objectives
By the end of this module, you will be able to:
- Explain reinforcement learning using a game analogy
- Identify the agent, environment, state, action, and reward
- Describe the core learning loop
- Understand the idea behind Q-learning
- See how RL differs from every other technique in this track
1. Learning Like a Gamer
Imagine playing a brand-new video game with no instructions. You press buttons, see what happens, and slowly figure out the rules: coins are good, cliffs are bad, jumping saves you. Nobody handed you a manual — you learned from consequences.
That's reinforcement learning (RL). Instead of learning from a labeled dataset, an agent learns by acting in an environment and collecting rewards.
The source curriculum sets this in the world of Peter and the Wolf: Peter (our agent) explores a grid world, trying to collect tasty apples 🍎 while avoiding the wolf 🐺. Nobody tells Peter the best path — he discovers it by trying, failing, and being rewarded.
Concept: Where supervised learning studies a fixed pile of labeled examples, reinforcement learning generates its own experience by interacting with a world. It learns from the results of its actions, not from an answer key.
2. The Five Ingredients
Every RL problem has the same cast of characters. Learn these five words and RL clicks into place:
| Term | What it is | In Peter's world |
|---|---|---|
| Agent | The learner/decision-maker | Peter |
| Environment | The world it acts in | The grid of ground, water, trees |
| State | The current situation | Where Peter is standing |
| Action | A choice the agent can make | Move up, down, left, right |
| Reward | Feedback on that action | +points for an apple, big penalty for the wolf |
3. The Core Loop
Reinforcement learning is a loop that repeats thousands of times:
- The agent observes its current state.
- It chooses an action.
- The environment returns a reward (good or bad) and a new state.
- The agent updates its strategy to favor actions that paid off.
- Repeat — getting a little wiser each time.
Early on, the agent flails and stumbles into the wolf constantly. But over many rounds, actions that lead to apples get reinforced and actions that lead to the wolf get avoided. Skill emerges from experience.
Concept: The agent's true goal isn't the next reward — it's the most reward over time. Sometimes that means a short-term sacrifice (a longer path) for a bigger payoff later (avoiding the wolf entirely). Balancing now vs. later is the soul of RL.
4. Q-Learning: A Cheat Sheet of Good Moves
So how does the agent remember what works? One classic method is Q-learning, which builds a kind of cheat sheet: for every state, a score for how good each possible action is (its "Q-value").
At first the cheat sheet is blank guesses. Each time Peter acts and sees the reward, he nudges the relevant score up or down. After enough exploring, the cheat sheet becomes a reliable guide: in this spot, moving right is worth a lot; moving toward the wolf is worth very little.
Did You Know? Reinforcement learning is behind some of AI's most famous moments — programs that beat world champions at Go and chess, robots learning to walk, and systems that fine-tune data-center cooling. The same "try, reward, repeat" idea that teaches Peter to dodge a wolf scales up to superhuman play.
5. Why RL Is Different
It's worth pausing on how unusual RL is compared to everything else you've learned:
| Supervised (Modules 4–7, 9–10) | Reinforcement (this module) | |
|---|---|---|
| Learns from | A fixed labeled dataset | Its own trial-and-error experience |
| Feedback | The "correct answer" | Rewards and penalties over time |
| Good for | Prediction & classification | Sequential decisions, control, games |
Try This! Think of a skill you learned by trial and error — riding a bike, cooking, a sport. What were your "rewards" (staying upright, a tasty meal) and "penalties" (falling, a burnt dinner)? Notice how you improved without anyone giving you a labeled dataset — you were doing reinforcement learning all along.
Key Takeaway: Reinforcement learning trains an agent to act in an environment, learning from rewards rather than labeled data. Through a loop of observe → act → reward → update, good behavior emerges over time. Q-learning stores a "cheat sheet" of how good each action is per state. It's how AI masters games, robotics, and any problem that's really a sequence of decisions.
This module is adapted from Microsoft's open-source ML-For-Beginners curriculum (MIT License). Reinforcement learning sketchnote by Tomomi Imura (CC BY-SA 4.0).