"Show a machine enough labeled examples, and it learns to sort the world into buckets — spam or not, cat or dog, Thai or Japanese."
Learning Objectives
By the end of this module, you will be able to:
- Explain what classification is and how it differs from regression
- Tell apart binary and multiclass classification
- Recognize the common classifier algorithms and their intuitions
- Understand why we try several classifiers and compare them
- Judge a classifier with the right kind of score
1. Sorting Into Categories
Classification predicts a category — a label — for each item. It's supervised learning, meaning we teach it with examples that are already labeled, then ask it to label new ones.
The source curriculum uses a delicious example: a dataset of world cuisines described by their ingredients. Given a handful of ingredients — say, coconut milk, fish sauce, and lemongrass — can a model guess the cuisine? (Thai, probably!) That's classification: from a list of clues (ingredients) to a class (cuisine).
You've actually met classification already: logistic regression in Module 5 was a classifier for two categories. Now we widen the lens.
Concept: Regression predicts a number ("how much?"). Classification predicts a category ("which one?"). Both are supervised — both learn from labeled examples — they just answer different kinds of questions.
2. Binary vs. Multiclass
How many buckets are we sorting into? That splits classification into two kinds:
| Binary | Multiclass | |
|---|---|---|
| Number of classes | Exactly 2 | 3 or more |
| Example | Spam / not spam | Thai / Indian / Japanese / Korean |
| Question | "Is it X or not?" | "Which of these is it?" |
Our cuisine problem is multiclass — many possible cuisines, and the model must pick the best fit from all of them.
3. A Toolbox of Classifiers
Here's a freeing truth: there is no single "best" classifier. Different algorithms shine on different data, so professionals try several and compare. Here are the classics you'll meet, each with a one-line intuition:
| Classifier | Intuition (how it decides) |
|---|---|
| Logistic Regression | Draws a boundary using probabilities (from Module 5) |
| K-Nearest Neighbors | "You are who your neighbors are" — copy the closest examples |
| Support Vector Machine | Finds the widest possible "street" separating the classes |
| Decision Tree | A flowchart of yes/no questions leading to an answer |
| Random Forest | A whole crowd of decision trees that vote on the answer |
Did You Know? Random Forest gets its power from a simple idea: a crowd is often wiser than any individual. It builds hundreds of slightly different decision trees and lets them vote. No single tree needs to be perfect — the majority tends to be right. It's one of the most reliable classical classifiers around.
Let's zoom in on the friendliest one. K-Nearest Neighbors (KNN) classifies a new item by looking at the handful of most similar examples it already knows. If you drop a new dish next to 5 known dishes and 4 of them are Thai, KNN says "Thai." It's intuitive precisely because it's how we often judge things — by comparison to what's nearby.
4. Building and Comparing
Because they all share Scikit-learn's fit / predict design, trying a new classifier is almost a copy-paste. Watch how little changes:
[object Object], sklearn.neighbors ,[object Object], KNeighborsClassifier
,[object Object], sklearn.svm ,[object Object], SVC
,[object Object],
knn = KNeighborsClassifier().fit(X_train, y_train)
svc = SVC().fit(X_train, y_train)
,[object Object],(,[object Object],, knn.score(X_test, y_test))
,[object Object],(,[object Object],, svc.score(X_test, y_test))This is exactly how a data scientist works: line up a few candidates, train them on the same data, and let the test scores decide the winner.
Pro Tip: Before trusting any classifier, make sure your data is balanced — that each class has enough examples. If 95% of your dishes are Italian, a model can score 95% by always guessing "Italian" while learning nothing. Balanced data (and honest metrics) keep you from being fooled.
5. Judging a Classifier
We measure classifiers with more than raw accuracy. Two especially useful ideas:
- Precision: when the model says "Thai," how often is it right?
- Recall: of all the truly Thai dishes, how many did it catch?
Together with the confusion matrix from Module 5, these tell you not just how often the model is right, but where it slips up — which is what you actually need to improve it.
Try This! Pick a sorting task from your life — sorting email into folders, tagging photos, filing receipts. Which classifier's intuition fits best? Would "judge by nearest neighbors" (KNN) work, or a "flowchart of questions" (decision tree)? There's no wrong answer — the goal is to feel how each algorithm thinks.
Key Takeaway: Classification predicts categories from labeled examples. It's binary (two classes) or multiclass (many), and there's no single best algorithm — logistic regression, KNN, SVM, decision trees, and random forests each have their own way of deciding, so we train several and compare. Balanced data and honest metrics (accuracy, precision, recall) tell us which one truly wins.
This module is adapted from Microsoft's open-source ML-For-Beginners curriculum (MIT License). Cuisine classification example by Jen Looper.