"To a machine learning model, the world is columns: the clues you feed in (features) and the answer you want out (label)."
Learning Objectives
By the end of this module, you will be able to:
- Define features and labels
- Understand feature engineering
- Pick useful inputs for a model
1. Features (X) and Labels (y)
- Features — the input clues (columns) the model learns from. Usually called X.
- Label — the answer you want to predict. Usually called y.
Key idea: Features are the questions ("how big? what color? what month?"); the label is the answer ("what price?"). A model learns the connection from features to label — so choosing good features matters more than almost anything else.
Explain like I'm new: Imagine guessing someone's age. The clues you use — height, voice, how they dress, the music they like — are features. The answer, their actual age, is the label. A model does exactly this: it studies clues (features) paired with known answers (labels) until it can guess the answer for someone new.
The letters X and y come straight from school algebra, where y = f(x) means "y depends on x." In machine learning it's the same idea scaled up: the label y is whatever you're trying to predict, and the features X are all the inputs it might depend on — often dozens or hundreds of columns rather than a single x. During training, the model sees many complete rows (features and their known labels) and works out the relationship. At prediction time you hand it only the features of something new, and it fills in the missing label. Keeping this X-in, y-out picture in your head makes almost every later ML idea easier to follow.
2. A Concrete Example
Predicting house price:
| Size (X) | Bedrooms (X) | Neighborhood (X) | Price (y) |
|---|---|---|---|
| 90 m² | 2 | Downtown | $320k |
| 140 m² | 3 | Suburb | $410k |
The model studies the features and their prices, then predicts the price of a new house from its features. Notice the shape: each row is one example (one house), each feature is a column, and the label sits in its own column as the thing you're trying to learn. Nearly every supervised ML dataset looks like this table.
One subtlety hides in that "Neighborhood" column: it holds words, not numbers, and most models can only do math on numbers. So text features usually get converted — for example, turning "Downtown / Suburb" into separate 0/1 columns the model can handle. You don't need the mechanics yet; just notice that features come in different flavors (numbers, categories, dates) and each may need a little preparation before the model can read it.
3. Feature Engineering
Feature engineering is creating better inputs — combining columns, extracting parts (day-of-week from a date), or scaling values — to help the model learn.
The reason this matters is that a model can only learn from what it can actually see in the columns. A raw timestamp like 2024-03-16 19:42 means little to an algorithm, but the features you extract from it — hour of day, weekend or not, is-it-a-holiday — line up neatly with real-world behavior the model can latch onto. Common moves include combining columns (price ÷ size = price-per-square-meter), extracting hidden parts (month or weekday from a date), and scaling so that a column measured in the millions doesn't drown out one measured in single digits. You're not changing the facts; you're re-packaging them into the clearest possible clues.
Simple example: From a raw "date" column you might create "is_weekend" — often far more predictive for, say, restaurant traffic than the raw date itself.
Real-world use case: A team predicting delivery times had a raw "pickup timestamp" that barely helped. By engineering features from it — hour of day, day of week, and an "is_rush_hour" flag — accuracy jumped. The model never understood time; the engineers translated a hard-to-use column into clues the model could actually learn from. That translation, not a fancier algorithm, was the win.
Common mistake: Throwing every column at the model "just in case." Irrelevant or leaky features add noise and can mislead it. Start with features you're confident matter, then add more only if they genuinely help.
Data scientist tip: Great features beat fancy algorithms. Time spent understanding your data and crafting meaningful features usually improves results more than swapping to a more complex model.
Key Takeaway: Models learn from features (input clues, X) to predict a label (the answer, y). Feature engineering — creating better inputs by combining, extracting, or scaling — often matters more than the choice of algorithm. Choose features you believe matter; don't dump in everything.
Further Learning
Part of the "Data Science and Data Scientist" course.