Clustering: finding the groups
Every model so far learned from examples that already had the answer filled in. Clustering is different: you hand it data with no answers at all, and it finds the natural groups hiding inside. We will use it to sort the World Cup teams into playing styles, all on their own.
Learning without an answer key
Remember from Lesson 1 that the label is the answer column. Logistic regression and decision trees both needed it: they are supervised learning, because the known answers supervise the training. Clustering throws the answer key away.
You know the answer for past examples (win or loss) and teach the model to predict it. Features in, a known label to learn from.
No answers at all. The model just finds structure, which points sit close together and which sit far apart, and forms groups called clusters.
Clustering sorts data into groups so that points in the same group are similar and points in different groups are not. Nobody says what the groups should be. The most popular method is called k-means, where k is just how many groups you ask for.
k-means, in four moves
k-means is almost embarrassingly simple. You pick how many groups you want (say k = 3), then repeat two steps until nothing changes.
"Nearest" just means the shortest straight-line distance. For two features, that is the same distance formula you may have met in maths class:
When the centres stop moving, the algorithm has converged and the groups are final. That is the whole method.
Group the World Cup teams yourself
Each dot below is a nation, placed by how many goals it scores per game (across the bottom) and how many it lets in per game (up the side). Attacking teams sit to the right; leaky teams sit high up. Nobody has labelled them. Run k-means and watch it discover the styles.
How to use this lab
- Press Drop 3 centres. The three big diamonds are guesses at where the group middles are.
- Press Next step. First press: every team takes the colour of its nearest diamond. Second press: each diamond slides to the middle of its colour group.
- Keep pressing Next step: assign, move, assign, move. Watch the diamonds settle.
- When they stop moving, k-means has converged. Read the three styles it found in the status line.
Why scouts love this
Clustering shows up all over football and beyond. A scouting department might cluster hundreds of players to find "cheap alternatives who play like our star." A streaming app clusters viewers to suggest shows. A shop clusters customers to send the right offer. Any time you want to find groups nobody defined in advance, clustering is the tool.
k-means does not tell you how many groups exist, you tell it. Ask for k = 3 and it finds three; ask for k = 5 and it finds five. Picking a sensible k is part of the skill, and often you just try a few and see which grouping makes the most real-world sense.
What k-means is really chasing
The four moves feel like a neat trick, but they are quietly hunting one number down. That number is the inertia (also called the within-cluster sum of squares): add up the squared distance from every point to the centre of its own group.
Read it slowly, left to right: for each of the $k$ groups $C_i$, take every team $x$ inside it, measure how far it sits from that group's centre $\mu_i$, square the distance, and total the lot. A small $J$ means tight, well-separated groups. k-means is simply trying to make $J$ as small as it can.
Now the two steps make sense. Assign each point to its nearest centre: that can only lower $J$, never raise it. Move each centre to the average of its points: the mean is the exact spot that makes the squared distance smallest, so that step lowers $J$ too. Two downhill moves, over and over, until there is nowhere lower to go. That is why it always settles.
That second line just says "the centre is the average of its members," worked out one axis at a time.
How many groups? The elbow method
You still have to pick $k$. The trick: run k-means for $k = 1, 2, 3, 4\dots$ and plot the inertia each time. More groups always lowers inertia (give every point its own centre and inertia hits zero), so the lowest point is not what you want. You want the elbow, the spot where adding one more group stops buying you much.
The same thing in real code
Libraries do every move for you. Here is the whole lab in a few lines of Python with scikit-learn, the toolkit you meet properly in the Python lesson. Press Run to see what it prints.
from sklearn.cluster import KMeans # the ready-made model
import numpy as np
# each row is one team: [goals scored, goals conceded] per game
X = np.array([[2.2, 0.6], [2.4, 0.7], [2.3, 0.8],
[1.4, 1.0], [1.3, 1.1], [1.5, 1.2],
[1.0, 1.6], [0.9, 1.8]])
model = KMeans(n_clusters=3, n_init=10, random_state=0)
groups = model.fit_predict(X) # assign + move until it settles
print("Group for each team:", groups)
print("Group centres:", model.cluster_centers_.round(2).tolist())
print("Inertia J:", round(model.inertia_, 2))Group for each team: [2 2 2 0 0 0 1 1] Group centres: [[1.4, 1.1], [0.95, 1.7], [2.3, 0.7]] Inertia J: 0.1
Same idea as the lab: pick k, let it assign and move until it converges, then read the groups. The three centres it prints are the attacking, balanced, and leaky middles you watched settle. And inertia_ is the very $J$ from the formula above.
- Clustering is unsupervised: it finds groups with no answer key.
- k-means repeats two steps: assign each point to its nearest centre, then move each centre to the middle of its group.
- It stops when the centres stop moving (it has converged).
- You choose k, the number of groups, and the elbow method helps you pick it.
- Under the hood it drives one number, the inertia, as low as it can go.
Next: the model loosely inspired by the brain, neural networks.