"Classification answers 'which one?' — sorting data into categories like spam/not-spam or cat/dog."
Learning Objectives
By the end of this module, you will be able to:
- Explain what classification predicts
- Tell binary from multiclass
- Recognize common classifiers
1. Predicting Categories
Classification predicts a category (a label), not a number. It's supervised learning where y is a class.
- Binary — two classes (spam / not spam)
- Multiclass — several classes (cat / dog / bird)
Key idea: Where regression draws a line to follow, classification draws a boundary to separate. The model learns where one category ends and another begins, then places new data on the right side.
Explain like I'm new: Picture sorting mail into labeled bins. Regression is like guessing the weight of each envelope (a number); classification is deciding which bin it belongs in (a category). The quick test: if the answer is one of a fixed set of buckets, it's classification.
The difference between binary and multiclass is just how many bins you're sorting into. Binary means two — spam or not, fraud or legitimate, will-churn or won't. Multiclass means three or more — classifying an animal photo as cat, dog, or bird, or tagging a support ticket as billing, technical, or general. A lot of the world's most valuable models are humble binary classifiers, because so many real decisions boil down to a yes-or-no: approve this loan? flag this transaction? show this ad? Don't assume "more classes" means "more advanced" — pick the number of bins your actual decision requires, no more.
2. Common Classifiers
| Classifier | Intuition |
|---|---|
| Logistic regression | Predicts the probability of a class |
| K-Nearest Neighbors | "You're like your closest neighbors" |
| Decision tree | A flowchart of yes/no questions |
| Random forest | Many trees voting together |
You try several and compare (Module 30). There's no universally "best" classifier — a decision tree might shine on one dataset while KNN wins on another, which is exactly why comparing a few is standard practice rather than a sign of indecision.
Each classifier has a different personality worth a mention. A decision tree is the most human-readable: it's literally a flowchart of yes/no questions ("odor present? → cap flat?"), so you can follow exactly why it decided what it did. K-Nearest Neighbors makes no grand theory at all — it just looks at the handful of most-similar past examples and copies their answer, like asking your closest neighbors what they'd do. Logistic regression draws a smooth probability-based boundary and is a reliable, fast baseline. And a random forest grows many different trees and lets them vote, which usually beats any single tree because their individual mistakes cancel out. Knowing these personalities helps you guess which might suit your problem before you even run them.
Simple example: Predicting whether a mushroom is edible or poisonous from its features (cap shape, color, odor) is classification — a real dataset in the source folder!
Real-world use case: A bank screens transactions as "fraud" or "not fraud" — binary classification. A photo app instead sorts pictures into "beach," "mountain," "city," and "indoors" — multiclass. Same underlying idea (learn a boundary between categories), scaled from two buckets to many.
3. Outputs Are Often Probabilities
Many classifiers don't just say "spam" — they say "92% likely spam." You choose a threshold (often 50%) to make the final call, which lets you tune how cautious the model is. Lowering the threshold catches more spam but risks flagging real mail; raising it does the opposite. That dial is a decision you make based on which mistake costs more.
Common mistake: Judging a classifier by accuracy alone. If 99% of emails are "not spam," a lazy model that always says "not spam" scores 99% while catching zero spam. Look at what kinds of mistakes it makes (Module 30).
Data scientist tip: Ask which error is more expensive before you tune anything. Missing a fraudulent charge and blocking a legitimate one are not equally bad — set your threshold to protect against the costlier mistake, not just to maximize a single number.
Key Takeaway: Classification is supervised learning that predicts a category — binary (two classes) or multiclass. Models (logistic regression, KNN, decision trees, random forests) learn a boundary between classes and often output probabilities. Don't judge on accuracy alone, especially with imbalanced classes.
Further Learning
Part of the "Data Science and Data Scientist" course.