A small one runs on this page. Watch it weigh its options, turn the one dial that changes its character, and send it down the roads it did not take.
Part one of three
Watch it choose
Give it the start of a sentence. Before every word, the model scores each of the 49,152 word-pieces it knows for how well it would fit next. These are the front-runners, with the odds it gives them.
The model never plans the sentence. It reads everything so far and produces one list of odds. One word is drawn from the list and added to the text. Then it reads the longer text and does it all again.
The pieces are called tokens. A token is usually a whole word, sometimes a fragment of one, sometimes a comma or a full stop.
Part two of three
Turn the dial
The word is drawn by chance, weighted by the odds. Temperature reshapes the odds before the draw. Turn it down and the favourite swells until nothing else can win. Turn it up and the list flattens until long shots get through.
The strip is the same list of odds, laid end to end. The marker is the draw: wherever it lands, that word is written. On this page the draws are held still, so everything that changes when you move the dial is the dial's doing.
Nothing inside the model changes. It gives the same scores every time. The “creativity” setting in an AI product is largely this one number, together with a cut-off like the forty-word shortlist used here.
Part three of three
Take the other road
Every word that lost was the start of a sentence that never got written. Pick any word below, then pick what it could have been instead. The model carries on from there.
An answer from a chatbot is one path through a tree like this one, only vastly larger. Ask again and you may be given a different path.
It is also why a model can say something false in the same fluent voice it uses for something true. Each word was likely, given the words before it.
Likely is not the same as right.