Two families of AI
Predictive (discriminative) models take an input and output a judgement about it: crab or fish, stock up or down, 32.4° tomorrow. Everything from Weeks 2, 3, and 5 lives here. The output is small: a label or a number.
Generative models output a whole new THING that resembles their training data: a paragraph that reads like human writing, an image that looks like a photo, audio that sounds like a voice. The output is huge: thousands of words or millions of pixels, assembled piece by piece.
Here is the beautiful secret that connects them: a generative model is a predictive model in a loop. A chatbot predicts one next token (a word piece), appends it, and predicts again, thousands of times. An image model predicts "slightly less noisy version of this image", hundreds of times. Generation = prediction, repeated. Your Week 5 skills did not expire. They got promoted.
When your best friend starts a sentence, you can usually finish it. If she says "Mi soon...", you already hear "come". You are not reading her mind. Your brain has heard thousands of her sentences and learned which words tend to follow which. A language model plays the same game, except it has "listened" to trillions of words, so it can finish almost anybody's sentence, in almost any style, on almost any topic.
Let's count an actual probability
Where do those next-token predictions come from? At heart, from counting patterns. Imagine a tiny training corpus of just 10 sentences, and every one of them starts with "The mango is":
- 5 of the sentences continue with "sweet"
- 3 continue with "ripe"
- 2 continue with "green"
Train a model on only that corpus, then ask it to finish "The mango is". Its predictions are exactly the counts turned into fractions:
A real model cannot literally store a count for every possible sentence, because most sentences it meets are brand new. Instead its neural network learns to ESTIMATE these fractions from patterns, with every weight tuned by gradient descent, exactly the Week 5 recipe. Same idea, industrial scale: trillions of words in, one probability for every token out.
Prediction sounds shallow, but think about what it takes to do it WELL. To predict the blank in "The capital of Jamaica is ___", the model must effectively know the fact (Kingston). To predict "The dogs ___ barking", it must have learned that "dogs" takes "are", not "is": that is grammar. To finish a physics answer, it needs some physics. Getting the next token right, billions of times, quietly forces the model to absorb spelling, grammar, facts, styles, and reasoning patterns. Nobody programmed those in. They are the price of winning the prediction game.
This belief is tempting because the output reads so smoothly that it FEELS looked-up. But the model's file is just billions of learned numbers (weights), thousands of times smaller than its training data, so the documents are not in there to copy from. Every reply is assembled token by token from probabilities. That is why a chatbot can write a poem about YOUR school that has never existed anywhere, and also why it can confidently state things that are wrong: nothing is being quoted, everything is being predicted.
The modality tour
The kind of data a model works with: text, images, audio, video, or code. Same prediction game, different playing field.
ChatGPT, Claude, Gemini. Trained on trillions of words to predict the next token. Powers chat, translation, code assistants, and your CXC study tools.
Midjourney, DALL-E, Stable Diffusion. Trained by adding noise to photos and learning to remove it. Lesson 3 is all about this trick.
Voice cloning, music generation (Suno, Udio), text-to-speech. Same next-piece prediction, on sound. Big questions for the music industry, including dancehall and soca.
Video models (Sora, Veo) generate moving scenes frame by frame. Code models write working programs, which is how your preteen siblings built games with Gemini this week.
🎛️ Sorting challenge: predictive or generative?
Tap each chip to sort it into the right family. Then check.
Why this is YOUR moment
Generative AI runs on three ingredients: data, computing power, and people who understand how to use both. The first two are concentrated in a few big companies. The third is wide open. A teenager in Kingston, Bridgetown, or Castries with Colab, a free Gemini account, and the understanding you are building this week can create software, content, and businesses that used to need whole teams. The Caribbean does not need permission to build with these tools. It needs builders who understand them. Hence this week.
Why is it fair to call a chatbot "a predictive model in a loop"?
1) Predictive models output judgements; generative models output new things. 2) Generation is prediction in a loop: next token, next denoising step, next note. 3) Every modality (text, image, audio, video, code) plays the same game on different data. Next lesson: the machine that made the text version world-changing, the transformer.
Next: transformers, attention, and the Token Beach game. 🏖️