Business Pill 42 · How a model is shrunk to fit
A model too large for a machine can be made smaller by storing its numbers with less precision. That is quantization.
A short explainer video. The numbers in it are round numbers for illustration.
The Short Answer
An engineer wants to run an AI model on her laptop. But the model needs 16 GB of memory, and the laptop has eight. It does not fit.
What the Model Is Made Of
Inside the model are billions of numbers, each one stored with great precision, such as 0.73281946.
So she rounds them. 0.73 is close enough. Each number now takes a quarter of the space.
Quantization, With Numbers
That rounding is called quantization: storing a model’s numbers with less precision, so it becomes smaller and faster.
In the video’s example, the model shrinks from 16 GB to 4. Now it fits on the laptop, and it answers faster.
The arithmetic uses the video’s round numbers. The 16 GB model is twice the laptop’s 8 GB, so it does not fit. A quarter of 16 GB is 4 GB, which is half of the laptop’s 8 GB, so it does.

The Price
The price is a little accuracy. Usually the loss is small, but on hard tasks it can show.
Why It Matters
So the question to ask is whether a smaller, cheaper model is good enough for this task.
The video’s advice in one line: round the numbers, shrink the model, then test what you lost.
The Question to Ask
- Is a smaller, cheaper model good enough for this task?
How This Concept Connects
Layer 2 of 8 · Models & Training, on the Business Engineer Concept Map
- Quantization builds on Parameters. Quantization stores the model’s numbers, its parameters, with less precision so it becomes smaller and faster.
- Quantization contrasts with Distillation. Both cut running cost: quantization rounds the same model’s numbers, distillation trains a smaller student on a teacher’s answers.
- On-Device AI (lesson unlocks soon) builds on Quantization. On-device AI needs a model small enough to live inside the phone, the shrinking that quantization performs.
The State of the Inference Economy
Quantization lowers inference cost and memory needs.Business Engineer framework
The Apple Silicon Disruption
Shrunken models enable on-device AI.
Get the full Concept Map as a free poster.
More Business Pills
- The Memory Wall: Why a Faster AI Chip Is Not Faster AI
- Tokens per Watt: What an AI Data Centre Actually Produces
- Why AI Can’t Be Both Instant and Cheap: Latency vs Throughput
- The Model and the Harness: Why Same-Model Products Differ
- Context, Not Capability: Why a Smart AI Model Gives Poor Answers
- Discardable Software: When Code Is Cheap Enough to Throw Away
- Human in the Loop vs Human on the Loop: Supervising AI Agents
- The AI Audit Problem: When Making Work Is Cheaper Than Checking It
- AI Evaluation as Acceptance Test: How to Know It’s Good Enough
- Extensibility Is Control: Who Holds the Power in an AI Product
- RLHF Explained: How an AI Model Learns What People Prefer
- Pretraining Explained: How an AI Model Learns Before Anyone Teaches It
- Fine-Tuning Explained: How to Adapt a General AI Model to One Job
- Tokens Explained: The Unit AI Reads, Writes and Bills In
- Embeddings Explained: How a Machine Compares Meaning
- RAG Explained: How a Model Answers From Your Documents
- AI Hallucination Explained: Why Models State False Things
- Distillation Explained: How a Small Model Learns From a Large One
- Reasoning Models Explained: What Changes When AI Thinks First
- Tool Use Explained: How an AI Model Goes From Text to Action
- Capex and Depreciation Explained: Why Chip Lifetime Drives AI Profits
- Run-Rate Revenue Explained: What an AI Company’s Number Means
- Backlog Explained: Revenue That Is Signed but Not Yet Earned
- Switching Costs Explained: Why It Is Hard to Leave an AI Supplier
- Temperature Explained: The Dial That Sets How Predictable an AI Model Is
- Guardrails Explained: Rules Enforced Around an AI Model, Not Inside It
- Benchmarks Explained: Why a Test Score Is Not Your Own Result
- Agent Memory Explained: How an AI Assistant Remembers You Between Chats
- MCP Explained: The Shared Plug That Connects AI Models to Tools
- Parameters Explained: Where an AI Model’s Knowledge Lives
- Scaling Laws Explained: Why AI Labs Keep Building Bigger
- Synthetic Data Explained: When a Model Writes Its Own Training Examples
- Transformers and Attention Explained: How AI Reads Every Word at Once









