Guessing the Next Word

Phone keyboards suggest the next word while you type. Language models do something similar, but with billions of patterns learned from text.

Given tokens so far, the model scores what token might come next. It picks one, appends it, and repeats until a stop condition.

Try the next-word lab. Change the sentence start and watch which words rise to the top of the list.

The model does not look up facts in a single table like a dictionary. It blends patterns: after Paris capital of people often write France.

That works well for common knowledge and grammar. It fails when patterns are wrong, rare or missing.

Temperature is a setting that controls randomness. Low temperature picks safe likely words. Higher temperature tries less common words for creative writing.

Clear prompts steer probabilities. If you ask for three bullet steps, likely continuations include list markers.

Asking the model to say I do not know when unsure can reduce confident nonsense on hard facts.

Next you will see where those patterns came from: training on huge text collections.