The same question put to an assistant twice will often return two different answers, sometimes differing in substance rather than only in wording. This is designed behaviour, and understanding it changes how the output should be used.
Generation is sampling, not lookup
At each step the model produces a probability across possible next tokens, and one is chosen from that distribution.
Always choosing the highest-probability token is possible and produces flat, repetitive text that loops on itself.
Introducing randomness produces text that reads naturally, and the price is that the same input does not yield the same output.
Small divergences compound
A different word early in a response constrains everything after it, because each subsequent token is conditioned on what came before.
An answer that begins by framing a question as a legal matter will develop differently from one that begins by framing it as a practical matter.
This is why variation is not evenly spread. Two responses are usually similar or substantially different, rarely halfway.
The temperature setting controls the spread
Products expose this as a temperature or creativity control, which reshapes the distribution before sampling.
Low values concentrate probability on the likeliest continuations and produce consistent, conservative output suited to extraction and classification.
Higher values flatten the distribution, which is useful for drafting and brainstorming and unhelpful when a stable answer is required.
Even a fixed setting does not guarantee repetition
Setting temperature to zero reduces variation sharply without eliminating it, because responses are computed in batches across shared hardware.
The arithmetic involved is not perfectly associative, so batch composition can alter results at the margin, and the batch depends on what other users are doing.
Providers also update models behind stable names, so an answer captured last month may not be reproducible at all.
What this means for relying on an answer
An output is one sample from a range of things the model might have said, not the model's position on the question.
Asking the same question several times and comparing is a cheap and underused check: broad agreement suggests the answer is well supported, while wide divergence signals that it is not.
Anything being used as a record — a decision, a classification, a figure entering a report — should be stored with its output rather than regenerated later, because the second run is a different sample.