What Is Machine Learning, Really?
Regular code: you write the rules. Machine learning: you give it examples with the answers, and it works out the rules itself. From a spam filter that breaks to features and labels, models as functions with knobs, generalization, the three kinds of learning (and how ChatGPT used all three), the ML workflow and what biased data does. Then real code: scikit-learn finds a rule I hid in made-up data, and a decision tree prints the rules it learned from the Iris flowers.
Chapters
- 0:00 Intro
- 1:16 Rules in, answers out
- 2:01 Examples, features, labels
- 2:38 A model is a function with knobs
- 3:43 The real goal: new examples
- 4:25 Supervised learning: regression and classification
- 4:58 Unsupervised learning
- 5:27 Reinforcement learning
- 6:00 ChatGPT used all three
- 6:33 The machine learning workflow
- 7:24 What can go wrong: garbage in, garbage out
- 8:35 Try it yourself
- 10:12 When not to use machine learning
- 10:48 Interview corner
- 11:53 Recap and next stop
Cheat sheet
- Regular code: rules + data → answers. Machine learning: data + answers → rules (a model)
- Example = one row · features = the inputs · label = the answer you want to predict
- A model = a function with adjustable numbers (parameters, the "knobs"); training finds settings that match the examples
- Supervised: labeled examples → regression (a number) or classification (a category)
- Unsupervised: no labels → find structure (clusters, unusual cases)
- Reinforcement: no answers, just rewards → learn by trial and error
- Workflow: data → split (hold out a test set, often about 20%) → train → evaluate → deploy → monitor → retrain
- The goal is generalization: doing well on examples the model has never seen
- Use ML when the rules are too many or too fuzzy to write, you have lots of good examples, and being wrong sometimes is OK
Code
Try it yourself (Python)
pip install numpy scikit-learn
import numpy as np
from sklearn.linear_model import LinearRegression
# Made-up deliveries. Hidden rule: 8 minutes + 3 per km, plus some randomness
rng = np.random.default_rng(1)
km = rng.uniform(1, 10, size=10).round(1)
minutes = (8 + 3 * km + rng.normal(0, 3, size=10)).round()
model = LinearRegression()
model.fit(km.reshape(-1, 1), minutes) # answers in...
print(f"learned: {model.coef_[0]:.2f} min per km + {model.intercept_:.2f}") # ...rule out
print("6.0 km:", model.predict([[6.0]])[0].round(1), "min")
# Real data: the Iris flowers
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier, export_text
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
iris.data, iris.target, test_size=0.2, random_state=0, stratify=iris.target)
tree = DecisionTreeClassifier(max_depth=2, random_state=0).fit(X_train, y_train)
print("test accuracy:", tree.score(X_test, y_test))
print(export_text(tree, feature_names=iris.feature_names, class_names=list(iris.target_names)))
What we got: 3.17 min per km + 6.52 (the hidden rule was 3 and 8), 25.5 min for 6 km, and 28 of 30 test flowers right (93.3%). Try changing the hidden rule, or the noise, and see how close the model gets.
learn_a_rule.pylearn_rules_iris.pyvisual_data.py
Sources
- Samuel, machine learning and checkers (IBM Journal, 1959)
- Paul Graham, A Plan for Spam (2002)
- Chollet, Deep Learning with Python, ch. 1 (the rules/answers framing)
- Reuters on Amazon's recruiting tool (Oct 2018)
- Gender Shades (Buolamwini & Gebru, 2018)
- AlphaGo (Silver et al., Nature, 2016)
- InstructGPT (Ouyang et al., 2022)
- Introducing ChatGPT (OpenAI, 2022)
- Fisher's Iris paper (1936; data by Edgar Anderson)
Transcript
Here's a task. Write a function that decides whether an email is spam. Easy, right? If the subject says free, it's spam. If it has three exclamation marks, spam.
If it says winner, spam. Then spammers write free with zeros. They put dots between the letters. You add more rules, and more, and the function never ends. And one day, a real email from a friend, asking if you want free tickets on Saturday, goes straight to spam.
Or, you could stop writing rules. Show a program thousands of emails that people already marked as spam or not spam, and let it work out the rules by itself. That's what the programmer Paul Graham did, back in 2002, after months of writing spam rules by hand. His new filter learned from his own mail. In his tests, it missed fewer than five spams in a thousand, and didn't flag a single good email.
That's machine learning. And "what is machine learning, and how is it different from regular programming?" is often the first question in an ML interview. Let's answer it properly.
This is Just Known. From unknown, to known. Episode two: what is machine learning, really? Here's the whole idea in one picture. In regular programming, you write the rules.
Data goes in, the rules run, and answers come out. Machine learning flips that around. You give it data and the answers you want, and it figures out the rules. Those learned rules are called a model. Then you feed the model new data, and it gives you new answers.
The name caught on in 1959, with a paper by Arthur Samuel at IBM, about a checkers program that got better by playing, even against itself. Nobody wrote down how to play well. It learned that from experience. Last episode, we looked down from the summit: ChatGPT. Today we start at the bottom of the climb.
So what does "data and answers" actually look like? Let's take a food delivery app that wants to tell you how long your order will take. It has thousands of past deliveries. Each one is an example. For each example, we know some facts up front: how far it went, how many items, whether it was raining.
Those inputs are called features. And we know how long it actually took. That's the answer we want to predict, called the label. To keep things simple, we'll use one feature: distance. Put the deliveries on a chart, distance across, minutes up, and you can almost see the rule.
Farther means longer. Now, a model. A model is just a function with some adjustable numbers inside it. Think of them as knobs. Here's a model with two knobs.
Minutes equals w times the distance, plus b. W is the minutes per kilometer. B is a base time, for cooking and pickup. Every setting of the two knobs draws a different line, and makes different predictions. Let's measure how wrong each setting is.
If we ignore distance and always guess the average, 24 minutes, we're off by about seven minutes on average. One minute per kilometer, plus ten? Even worse: almost nine. Two per kilometer: under four. Keep turning the right way, and you get to 3.17 minutes per kilometer, plus 6.52.
Now the average miss is under one minute. That's what training is: searching for the knob settings that make the predictions match the labels. How the computer does that search, efficiently, is the whole of episode three. And remember ChatGPT's billions of knobs from episode one? Same idea.
Just a lot more knobs. But fitting the examples you have isn't the goal. A lookup table can do that. Store all ten deliveries, and it's right every time, on deliveries that already happened. The goal is the next order.
Six kilometers? The lookup table has nothing. The line says about 25 and a half minutes. Doing well on examples you've never seen is called generalization, and it's the whole point. A model that only memorizes hasn't learned anything.
Here's a twist. On these ten deliveries, the learned line is off by less than the hidden rule I used to make the data. It's matching a bit of the random noise, too. A little of that is harmless. A lot of it is called overfitting, and it gets its own episode: episode five.
Everything so far is supervised learning: every example comes with its label, the right answer, like a teacher marking the work. It comes in two flavors. When the answer is a number, like minutes, a price, or tomorrow's temperature, it's called regression. When the answer is a category, like spam or not spam, cat or dog, it's called classification. Same recipe either way: examples with labels go in, a model comes out.
Classification gets its own deep dive in episode four. Sometimes you have plenty of data but no labels at all. That's unsupervised learning. Nobody tells the model the answers. It looks for structure on its own.
Give it every customer's order history, and it can find groups: say, late-night snackers, family dinners, office lunches. Nobody defined those groups in advance. It can also spot the one order that doesn't fit any pattern, which might be fraud. We'll do this for real in episode eight. The third kind is reinforcement learning.
There are no right answers to copy here, just a score. The model acts, gets a reward or a penalty, and learns over many tries which actions pay off. It's a big part of how computers learned to play games better than people. In 2016, DeepMind's AlphaGo beat Lee Sedol, one of the world's best Go players, four games to one. It learned from human games first, then got stronger by playing against itself, and searched ahead before every move.
Episode thirteen. These three kinds aren't rival camps. Real systems mix them, and ChatGPT is a good example. Its first training stage, predicting the next token, is a clever twist on supervised learning. The labels come free: in any sentence, the next word is the answer.
That's called self-supervised learning, and it's why the model can learn from so much text. Then it's fine-tuned on example conversations written by people, which is plain supervised learning. Then people rank its answers, and reinforcement learning pushes it toward the ones they preferred. The full story is in episode seventeen. So how does a real project go?
Almost always the same way. First, data: collect examples and clean them up. That's where most of the work goes. Then, split them. Lock away a slice, often about a fifth, as a test set, and never train on it.
Train on the rest. Then evaluate the model on the test set. Think of it as the exam: questions the model has never seen. Testing on the training data would be like grading a student on the exact questions they practiced. If it passes, deploy it, and the app starts showing real delivery times.
Then keep monitoring it, because the world changes. A rainy season, new restaurants, a new part of town, and the old patterns stop fitting. When the misses grow, you collect fresh data and train again. So what can go wrong? A model learns whatever patterns are in its examples, including the ones you didn't want.
Garbage in, garbage out. In 2018, Reuters reported that Amazon had built an experimental tool to rate job applicants. It learned from ten years of résumés sent to the company, most of them from men. So it taught itself that male candidates were preferable. It downgraded résumés that included the word women's, as in women's chess club captain.
Amazon shut the project down. Reuters reported that recruiters had looked at its rankings, but never relied on them alone. That same year, a study called Gender Shades tested three commercial systems that guess a person's gender from a photo. On lighter-skinned men, they were wrong less than one percent of the time. On darker-skinned women, up to thirty-five percent.
Nobody wrote a biased rule. But the popular face datasets the study looked at were mostly lighter-skinned faces, and a model can only learn from what it sees. The model doesn't know what's fair. It only knows its examples. So check what's in the data, test the model on every group it'll be used on, and keep a human in the loop for decisions that matter.
You can try this yourself, with Python and scikit-learn, a free, open-source library that's a standard first tool for machine learning. The code is in the description. I made up ten deliveries with a hidden rule: eight minutes, plus three per kilometer, plus some randomness for traffic. The model never sees that rule. It only sees the ten examples.
Two lines do the learning. Create the model. Call fit with the features and the labels. Answers in. Then ask what it learned: 3.17 minutes per kilometer, plus 6.52.
Rule out. That's close to the hidden rule, but not exact, because ten noisy examples can't pin it down perfectly. More data would get closer. And predict gives the answer for a new order: six kilometers, about 25 and a half minutes. Now real data: the Iris flowers, measured in the 1930s and probably the most famous dataset in machine learning.
A hundred and fifty flowers, three species. The features are four measurements of each flower's petals and sepals. The label is the species. Split it: a hundred and twenty flowers to train on, thirty locked away for the test. This time the model is a decision tree, a flowchart of yes-or-no questions that it builds by itself.
On the thirty test flowers, it gets twenty-eight right. Good, not perfect. And here's my favorite part: you can print the rules it learned. If the petal is narrower than 0.8 centimeters, it's a setosa. Otherwise, under 1.75, versicolor.
Wider than that, virginica. Nobody wrote those rules. It found them in the examples. So when should you reach for machine learning? Not by default.
If you can write the rules, write the rules. Tax calculations, sorting, checking a password's length: regular code is exact, cheap, and easy to test. Machine learning earns its place when the rules are too many or too fuzzy to write down, like spotting spam or a face, when they keep changing, and when you have lots of good examples. And you have to be fine with being wrong sometimes. Even our tidy flower model missed two in thirty.
Interview corner. What is machine learning, and how is it different from regular programming? Pause here, and try answering in thirty seconds.
Here's a strong answer. In regular programming, you write the rules, and the program applies them to data. In machine learning, you give it examples, data with the answers, and it learns the rules itself, as a model with parameters fitted to the data. There are three main kinds. Supervised learning learns from labeled examples, to predict a number, which is regression, or a category, which is classification.
Unsupervised learning finds structure without labels, like clusters. Reinforcement learning learns from rewards, by trial and error. You train on part of the data, and evaluate on a held-out test set, because the real goal is to generalize to examples the model has never seen. To stand out, add when you wouldn't use it: if the rules are clear and stable, regular code is simpler, cheaper, and exact. Let's recap.
Regular code is rules in, answers out. Machine learning is answers in, rules out. A model is a function with knobs, and training finds the knob settings that best match the examples. And the real test is always new data the model has never seen. Today we turned the knobs by trying settings until the misses shrank.
But a real model can have millions of knobs, or billions, and you can't just try them all. So how does it find the right settings? It walks downhill, blindfolded. That's gradient descent, in episode three. This is Just Known.
From unknown, to known.
Note: the delivery data is made up from a hidden rule (as the video says); every model number on screen is real output from the code above, and the Iris data is real. Pictures marked "illustration" are not data. ChatGPT is a trademark of OpenAI; this channel isn't affiliated with OpenAI.
Narration voice generated with Google Gemini text-to-speech.
