Train a Network, Live

The setup

The data: 30 daily maximum temperatures (a realistic Kingston July: warm, a gentle warming trend, a rainy-day dip). The task: given 3 consecutive days, predict the next day. That gives us 27 training examples from 30 days. We hold out the last 6 examples as a validation set: the network never trains on them, so they tell us honestly whether it learned the pattern or memorised the answers.

The network: 3 inputs → a hidden layer you size yourself → 1 output. Exactly the Lesson 1 machinery, trained with exactly the Lesson 2 algorithm.

What actually happens when you press Train

No magic. Every epoch, the network runs the same four-step loop from Lessons 1 and 2 on each of the 21 training examples, dozens of times per second:

The nudge is the Lesson 2 update rule, applied to every weight in the network at once:

One worked example so the numbers feel real. Take a one-weight network where prediction = weight × input. The weight is 2, the input is 2, and the true answer is 6. The forward pass gives 2 × 2 = 4, so the error is 4 − 6 = −2. For squared loss the gradient is 2 × error × input = 2 × (−2) × 2 = −8. With a learning rate of 0.05, the update is 2 − 0.05 × (−8) = 2.4. Next forward pass: 2.4 × 2 = 4.8, which is closer to 6 than 4 was. One small improvement. Repeat it for every weight, on every example, for hundreds of epochs, and you get the show you are about to watch: the loss sliding downhill and the orange line locking onto the dots.

🔬 The lab

Troubleshooting: when training looks wrong

• Loss barely moves. Either the steps are too small or the network is too tiny to capture the pattern. Raise the learning rate one notch, or add hidden neurons, then reset the weights and train again.
• Loss jumps around wildly or explodes. The learning rate is too big, so every step leaps clean over the valley instead of walking down into it. Drop it to 0.05 or 0.005 and retrain.
• Train loss is tiny but validation loss is huge. The network memorised its 21 training examples instead of learning the weather pattern. Use fewer hidden neurons, stop training earlier, or get more data.
• Both curves fall, then the orange one turns back up. That upturn is the exact moment overfitting begins. The best network was the one at the bottom of the orange curve, so stopping there (early stopping) is a real technique the pros use every day.

🐍 The same model in Keras

Everything you just did by hand is three lines in Keras, the friendly deep learning library you will use in your final assignment on Google Colab (free, runs in the browser, nothing to install; you used it in Week 2). Read it and notice you can now explain EVERY line:

import numpy as np
from tensorflow import keras

# 30 days of Kingston max temperatures (°C)
temps = np.array([30.1, 30.4, 29.8, 30.6, 31.0, 30.7, 31.2,
                  31.5, 30.9, 28.4, 29.1, 30.8, 31.4, 31.6,
                  31.2, 31.8, 32.0, 31.5, 31.9, 32.2, 31.7,
                  30.2, 30.9, 31.8, 32.1, 32.4, 32.0, 32.5,
                  32.3, 32.6])

# windows of 3 days -> next day
X = np.array([temps[i:i+3] for i in range(27)])
y = temps[3:]

model = keras.Sequential([
    keras.layers.Dense(8, activation="relu", input_shape=(3,)),
    keras.layers.Dense(1)
])
model.compile(optimizer=keras.optimizers.Adam(0.01), loss="mse")
model.fit(X, y, epochs=200, validation_split=0.2, verbose=0)

tomorrow = model.predict(temps[-3:].reshape(1, 3), verbose=0)
print(f"Day 31 prediction: {tomorrow[0][0]:.1f} °C")

Line by line: Dense(8, activation="relu") is your hidden layer of 8 Lesson-1 neurons. loss="mse" is the Lesson-2 wrongness measure. Adam(0.01) is gradient descent with a 0.01 learning rate (plus momentum). epochs=200 is 200 passes through the data, and validation_split=0.2 holds out 20% to catch overfitting. You know all of this now, for real.