Where deep learning sits in your journey
Quick map of everything you have done this summer. Week 1-2: statistics and classic machine learning (averages, logistic regression, decision trees). Week 3: prediction on real financial data. Those models are powerful but shallow: they learn ONE equation or ONE set of splits. Deep learning stacks many simple learners into layers, and the layers learn to build on each other. That stack is a neural network, and "deep" literally means "many layers".
Why does that matter? Because layered learning discovers its own features. In Week 3 YOU decided the model should look at "yesterday's price" and "the 5-day average". A deep network, given enough data, invents its own versions of those, including ones no human thought of. That is why it powers speech recognition, hurricane track forecasting, medical imaging, and every chatbot you have used.
The neuron: a three-step calculator
A neuron takes numbers in and pushes one number out. Three steps:
Concrete example. A neuron guesses if a football team will score, from two inputs: shots taken (x1 = 12) and shots on target (x2 = 5). Suppose the weights are w1 = 0.2, w2 = 0.8, and bias b = −4:
Notice the weights encode judgement: shots on target (0.8) matter four times more than mere shots (0.2). The bias (−4) sets the bar the evidence has to clear. Training is nothing more than the network finding good values for every weight and bias.
Rerun the example with no bias. The sum becomes 2.4 + 4.0 = 6.4, and sigmoid(6.4) ≈ 0.998: the neuron screams "they will score!" at almost any input. The bias of −4 is a demand for evidence: the weighted inputs must add up to more than 4 before the neuron even reaches a 50-50 verdict. Push the bias toward 0 and the neuron gets easier to convince; push it further negative and it turns skeptical. Weights decide WHAT counts as evidence, the bias decides HOW MUCH evidence is enough.
Activation: the neuron's attitude
Step 3 passes the sum through an activation function. Why not just output 2.4 directly? Because stacking plain sums gives you… one big sum. Linear in, linear out, no matter how many layers. The activation adds a bend, and bends are what let networks learn curves, edges, and rhythms. The two you will meet constantly:
"If the sum is negative, output 0. Otherwise pass it through." Brutally simple, and it is the default inside modern networks. ReLU(2.4) = 2.4, ReLU(−1.3) = 0.
Squashes any number into 0 to 1, perfect for probabilities. Sigmoid(2.4) ≈ 0.92, so "92% chance they score". You met this in logistic regression, Week 2!
Look again at the sigmoid neuron: weighted sum, plus bias, squashed into a probability. That is EXACTLY the logistic regression you trained in Week 2. One neuron with a sigmoid activation IS logistic regression. So a deep network is not an alien machine: it is layers of Week 2 models wired together, each one passing its verdict to the next as a fresh input. If you understood Week 2, you already understand every individual part of this lesson; the new idea is only the stacking.
🎛️ Interactive: drive a real neuron
This is the exact neuron from the example. Drag the sliders and watch every step recompute. Challenge: find weights where 12 shots and 5 on target give a probability BELOW 30%.
A neuron has weights w₁ = 0.5, w₂ = 1.0, bias b = −2, with inputs x₁ = 2, x₂ = 3. What is the weighted sum plus bias?
From neurons to layers to deep
One neuron draws one straight boundary (the preteens literally drag that line in their course). The power move is composition:
- Input layer: your raw numbers. For weather: the last 7 days of temperatures. For football: shots, possession, form, ranking.
- Hidden layers: each neuron in layer 1 learns a mini-pattern from the inputs. Each neuron in layer 2 learns patterns OF those patterns. Stack more layers, get more abstraction.
- Output layer: the answer. One neuron for "tomorrow's temperature", or three for "home win / draw / away win" probabilities.
A network for our weather task might be 7 inputs → 16 hidden neurons → 8 hidden neurons → 1 output. Tiny by industry standards (GPT-class models have hundreds of billions of weights), but the machinery is identical.
Any number the network learns for itself: every weight and every bias. The football neuron above has 3 parameters (w₁, w₂, b). The 7 → 16 → 8 → 1 weather network has 273 (count every connection plus every bias). GPT-class models have hundreds of billions. Different scale, exact same idea: knobs the training algorithm gets to turn.
Follow one prediction through a whole network
The diagram stays abstract until you push numbers through it, so let's run a mini version by hand: 2 inputs, a hidden layer of 2 ReLU neurons (call them A and B), then 1 sigmoid output. Inputs: x1 = 2, x2 = 1.
Two things to notice. First, the hidden neurons' OUTPUTS became the output neuron's INPUTS: that hand-off is all the word "layers" means. Second, ReLU switched neuron A off completely, so the final verdict came from B alone. Feed in a different x1 and x2 and A might wake up while B goes quiet. That switching is the payoff of the activation's bend: one network can behave like many different simple models, choosing which of its parts speak for each input.
Think of a sound engineer's mixing board. Each slider (weight) controls how much of one input makes it into the mix. The bias is the master volume offset, and the activation is the limiter that shapes the final signal. Training a network = a very patient engineer adjusting thousands of sliders until the output sounds exactly right, except the "engineer" is an algorithm, which is Lesson 2.
The word "neuron" makes this easy to believe, and headlines lean into it. But you just computed everything a neuron does: multiply, add, squash. No thoughts, no wants, no understanding. A network with billions of parameters is billions of multiply-add-squash steps, tuned by the algorithm you meet in Lesson 2. Hold on to that picture and AI news becomes much easier to judge: the useful question is never "what does it think?", it is "what was it trained to predict, and on what data?"
1) A neuron computes activation(weights × inputs + bias). 2) Activations like ReLU and sigmoid add the bends that make learning curves possible. 3) Layers of neurons build features out of features, and "deep" = many layers. 4) Training means finding good weights and biases, which is the next lesson.
Next: how the network finds those weights. Bring your hiking boots. ⛰️