How AI Actually Works

Ask an AI the same question twice, in two separate conversations, and you'll sometimes get two different answers. Not contradictory garbage — just different. Different word choice, different examples, occasionally a different conclusion on a genuinely ambiguous question. If your mental model of AI is "a really smart search engine" or "a database that talks," that behavior looks like a bug. It isn't. It's the single most important fact about how these systems work, and once it clicks, a dozen other confusing behaviors stop being confusing.

It predicts the next token — that's the whole mechanism

Strip away the marketing and a language model does one thing, over and over: given all the text so far, predict the single most likely next chunk of text, called a token (roughly a word or word-fragment — "trading" might be one token, "unstoppable" might split into "un" + "stop" + "able"). It picks a token, appends it to what came before, and then runs the exact same prediction again, now with one more token of context to work with.

That's it. There's no separate "understanding" step, no "let me check my notes" pause in the middle. The apparent intelligence is what emerges from doing that one narrow trick — predict the next likely piece of text — extremely well, after being shown a training set large enough to absorb the patterns of how language expresses reasoning, code, arguments, and explanations.

Look at what that diagram shows: at every step, the model re-reads everything generated so far — including its own previous output — and predicts one more piece. There is no finished sentence sitting behind the scenes waiting to be revealed. The "plan" you feel behind a good answer is an illusion created by a very good next-piece predictor running many times in a row.

Training built a pattern-machine, not a filing cabinet

"Training" is the process that produced this next-token predictor, and it does not work the way most people assume. The model was not loaded up with a giant table of facts that it now looks up on demand. During training, it was shown enormous amounts of text and adjusted, incrementally, so its next-token predictions got closer to what actually appeared in that text. What comes out the other end is a huge set of statistical patterns about how language flows — not a stored copy of any particular sentence, and not an indexed record of any particular fact.

This is the part that trips people up the most, so it helps to see it side by side against something everyone already understands: a database.

A database gives you back exactly what was stored, every time, or tells you cleanly that it found nothing. A model reconstructs a plausible-sounding answer from patterns — even when nothing resembling that exact answer exists anywhere in what it was trained on. That's not a defect waiting on better engineering; it's what "prediction" means as opposed to "retrieval." Tools like web search or a connected database can be bolted onto a model to give it a real lookup step — that's what's actually happening when an AI product "searches the web" for you mid-conversation. Without that bolted-on step, the model is working from absorbed patterns alone, with nothing underneath to check itself against.

Why the same question gets different answers

Here's where sampling comes in, and it explains the identical-question-different-answer behavior from the top of this lesson. At each step, the model doesn't compute one single "correct" next token — it computes a probability across many candidate next tokens. Maybe "profitable" is 40% likely to come next, "risky" is 25% likely, "volatile" is 15% likely, and a long tail of other words splits the rest.

A setting called temperature controls how the model chooses among those candidates. At temperature zero, it always takes the single highest-probability token — same input, same output, every time, like a very expensive calculator. Turn temperature up, and it starts occasionally picking the second- or third-most-likely token instead of always the top one, which is exactly a weighted dice roll across that shortlist. Most AI products run with temperature above zero on purpose, because a little variation produces more natural, less repetitive writing. That one dial is the entire explanation for why you got a different — not wrong, just different — answer the second time you asked.

What this actually makes models good and bad at

Once you see the mechanism, the strengths and weaknesses stop being mysterious.

Genuinely good at: transforming text — summarizing, rewriting, translating, reformatting; generating plausible first drafts of almost anything language-shaped, including code, emails, explanations, and plans; and pattern-completing tasks where "what usually comes next in text like this" is a good guide to what should come next in your specific case.

Genuinely bad at: anything requiring one exact, single correct fact it has to retrieve rather than reconstruct — a specific statistic, a specific citation, a specific version number; anything requiring true novelty outside the patterns in its training data; and, the one that causes the most real-world damage, telling the difference between "I have strong evidence for this" and "this sounds like the kind of thing that's usually true." Both come out in the identical, confident tone. That last gap is exactly what the "Trust, But Verify" lesson later in this track is built around.

Bottom line

An AI model doesn't know things the way a database knows things — it predicts things, one token at a time, based on patterns absorbed during training, with a deliberate dose of randomness mixed into which token it picks. That's why identical questions can get different-but-plausible answers, why it can sound completely certain while being completely wrong, and why the tasks it's best at are the ones where "a fluent, pattern-consistent answer" and "a correct answer" happen to be the same thing. Next up: the context window — the other half of the mechanism, and the reason AI seems to forget things you told it five minutes ago.