Get the 2026 ML Training Cookbook | 52 recipes — GRPO, Flow Matching, World Models, and everything in between Download Now →
Temperature Scaling
A parameter that controls the randomness of model predictions by scaling logits before softmax, trading determinism for creativity.
Temperature Scaling is a technique that controls the randomness or 'creativity' of a language model's output by dividing the logits (pre-softmax scores) by a temperature parameter T before applying the softmax function. A temperature of 1.0 leaves the distribution unchanged. Temperatures below 1.0 (e.g., 0.2) sharpen the distribution, making the model more deterministic and likely to select high-probability tokens. Temperatures above 1.0 (e.g., 1.5) flatten the distribution, increasing randomness and diversity.
Temperature is one of the most important inference-time hyperparameters. Low temperatures are suitable for tasks requiring precision and consistency, such as code generation, factual question answering, and data extraction — where the model should always select the most likely next token. High temperatures are useful for creative tasks like brainstorming, story generation, and poetry, where diversity and novelty are valued over precision.
In the context of knowledge distillation, temperature scaling serves a different purpose: it softens the teacher's output distribution to reveal 'dark knowledge' — the relative probabilities the teacher assigns to non-target classes. A higher temperature during distillation makes the student learn from the richer signal in the teacher's full distribution rather than just the top prediction. The temperature used during distillation training is typically compensated for by squaring the temperature factor in the loss function.
- ■Divides logits by temperature T before softmax; T=1.0 is unchanged
- ■T<1.0 sharpens distribution (more deterministic); T>1.0 flattens it (more random)
- ■Low temperature for precision tasks; high temperature for creative tasks
- ■Also used in knowledge distillation to soften teacher distributions
- ■One of the most impactful inference-time hyperparameters
Setting temperature to 0.1 for a code-generation model to ensure deterministic, high-probability outputs, versus 0.9 for a creative writing assistant.