Temperature Explained: The Dial That Sets How Predictable an AI Model Is

Business Pill 37 · The dial that sets how predictable a model is

Ask a model the same question twice and you can get the same answer or a different one. The dial that sets which is called temperature, and the right setting depends on the job.

A short explainer video, under a minute. It uses a story about a robot that writes slogans for a coffee shop.

The Short Answer

A robot writes slogans for a coffee shop. The owner asks for a slogan, and the robot says “Great coffee.” Every day she asks again, and every day she gets the same line.

So an engineer reaches for a dial inside the robot and turns it up. Now every answer is different. Then the answers get a little too different, and one of them is “Fly banana fly.” That dial has a name. It is called temperature.

What Temperature Controls

When a model writes, it picks one word after another. Temperature says how much chance the model allows when it picks the next word.

The coffee shop shows both ends of the dial. With too little chance, the owner gets the same slogan every day. With too much, she gets a banana.

Low and High Temperature

At low temperature, the model gives the same safe answer every time. That is good for facts, and for anything a customer relies on.

At high temperature, the model gives fresh answers and more surprises. That is good for ideas and first drafts.

Why It Matters

Temperature is a choice about the task, not a measure of how good the model is. The same model can be set to stay put or to explore.

So the rule from the video is short: low for facts, high for ideas. Answers a customer relies on should stay the same. Ideas and first drafts benefit from variety.

The One Question to Ask

1. Does this task need the same answer every time, or a new one?

More Business Pills

• The Memory Wall: Why a Faster AI Chip Is Not Faster AI
• Tokens per Watt: What an AI Data Centre Actually Produces
• Why AI Can’t Be Both Instant and Cheap: Latency vs Throughput
• The Model and the Harness: Why Same-Model Products Differ
• Context, Not Capability: Why a Smart AI Model Gives Poor Answers
• Discardable Software: When Code Is Cheap Enough to Throw Away
• Human in the Loop vs Human on the Loop: Supervising AI Agents
• The AI Audit Problem: When Making Work Is Cheaper Than Checking It
• AI Evaluation as Acceptance Test: How to Know It’s Good Enough
• Extensibility Is Control: Who Holds the Power in an AI Product
• RLHF Explained: How an AI Model Learns What People Prefer
• Pretraining Explained: How an AI Model Learns Before Anyone Teaches It
• Fine-Tuning Explained: How to Adapt a General AI Model to One Job
• Tokens Explained: The Unit AI Reads, Writes and Bills In
• Embeddings Explained: How a Machine Compares Meaning
• RAG Explained: How a Model Answers From Your Documents
• AI Hallucination Explained: Why Models State False Things
• Distillation Explained: How a Small Model Learns From a Large One
• Reasoning Models Explained: What Changes When AI Thinks First
• Tool Use Explained: How an AI Model Goes From Text to Action
• Capex and Depreciation Explained: Why Chip Lifetime Drives AI Profits
• Run-Rate Revenue Explained: What an AI Company’s Number Means
• Backlog Explained: Revenue That Is Signed but Not Yet Earned
• Switching Costs Explained: Why It Is Hard to Leave an AI Supplier
• Guardrails Explained: Rules Enforced Around an AI Model, Not Inside It
• Benchmarks Explained: Why a Test Score Is Not Your Own Result
• Agent Memory Explained: How an AI Assistant Remembers You Between Chats
• MCP Explained: The Shared Plug That Connects AI Models to Tools
• Quantization Explained: How a Large AI Model Is Shrunk to Fit
• Parameters Explained: Where an AI Model’s Knowledge Lives
• Scaling Laws Explained: Why AI Labs Keep Building Bigger
• Synthetic Data Explained: When a Model Writes Its Own Training Examples
• Transformers and Attention Explained: How AI Reads Every Word at Once

Business Engineer

The Map of AI

The Business Engineer framework maps where each part of the AI stack sits and who holds the leverage. These pills are short lessons from the same library.

Business Engineer

the Map of AI — 200+ companies across 9 layers

Read →

Business Engineer

the AI value chain — a multi-layered analysis

Read →

Explore the Map of AI →