LSTM Neural Net
Trees look at a row of features and forget everything else. But prices, and bills, come as a sequence in time, where what happened months ago can still matter. The LSTM is a neural network built for exactly this: it carries a memory forward through a sequence, choosing what to keep and what to let go.
LSTM stands for Long Short-Term Memory
The name is the idea. It is a neural network that reads a sequence one step at a time and keeps a running memory (called the cell state). At every step, three little gates decide what happens to that memory:
Here is the same cell drawn across three time steps. The thick teal arrow is the memory flowing forward through time; at each step the gates edit it, and an answer pops out the top.
The cell state is a value the LSTM carries from step to step down the whole sequence. The forget gate is what makes it special: it can hold onto an important clue for a long time, or wipe it the instant it stops mattering.
The memory test
Imagine a year of light bills. December was huge (holiday lights, everyone home). Every month after is calm. Question: by next December, does the model still remember last December? Drag the forget gate and find out. High memory keeps December alive for months; low memory forgets it almost at once.
How to play
- Drag the forget gate from low (forgetful) to high (holds on).
- Watch the gold memory line: with a high gate the December spike lingers; with a low gate it vanishes by spring.
- Press Play the sequence to watch the memory update month by month.
What it is great at, and where it slips
Long sequences where distant history matters: language, music, and long stretches of prices. It can learn patterns a single tree cannot see.
Data is small. LSTMs are hungry for examples and computing power, and on a short, noisy series they often lose to a simple model.
The LSTM mindset is about memory that matters. Last December's spike, or last summer's heatwave, is a clue worth carrying forward when you guess this year's bill. You do not need a neural network to use that idea, and you will in the bonus light-bill challenge.
- An LSTM is a neural network that reads a sequence and carries a memory forward.
- Its forget gate decides how long to hold onto old information.
- It shines on long sequences but needs lots of data, so simple models often win on short ones.
Next: write the statistical models in real Python code.