Business Pill poster: rate limits

Rate Limits Explained: Why an AI Product Can Stall in Its Busiest Minute

Business Pill · Only so many per minute

A rate limit is a cap on how many requests a customer may send each minute. For an AI product it means the busiest minute is capped by someone else’s capacity.

A short explainer video, under a minute. The restaurant and the doorman are an illustration.

At a glance

What a Rate Limit Is

On launch day, a company's new app sends more requests than its allowance, and the answers stop.

What It Means for a Product

The video's board puts it in one line: it means your busiest minute is capped by someone else's capacity.

Why It Matters

The video turns this into one question: what happens to our product in its busiest minute?

The Short Answer

Picture a restaurant with one kitchen. At eight o’clock everybody orders at once, so the door lets in only so many tables per minute. The rest wait outside.

AI providers do the same. Their chips are the kitchen.

What a Rate Limit Is

On launch day, a company’s new app sends more requests than its allowance, and the answers stop.

That allowance is called a rate limit: a cap on how many requests a customer may send each minute.

What It Means for a Product

The video’s board puts it in one line: it means your busiest minute is capped by someone else’s capacity.

The busiest minute is the one that matters most, and it is the one the cap bites hardest.

The Idea in Parts

The lesson in its own parts

What a Rate Limit Is

On launch day, a company's new app sends more requests than its allowance, and the answers stop.

What It Means for a Product

The video's board puts it in one line: it means your busiest minute is capped by someone else's capacity.

The Ways Out

The video lists three ways out: reserve capacity in advance, queue the work that can wait, or keep a second provider ready.

The Ways Out

The video lists three ways out: reserve capacity in advance, queue the work that can wait, or keep a second provider ready.

Why It Matters

The video turns this into one question: what happens to our product in its busiest minute?

Computing power is shared and finite. Know your limit before your launch.

The key insight: Computing power is shared and finite. Know your limit before your launch.

Where This Shows Up in AI Business Today

From our news coverage

OpenAI Says It Disrupted a Distillation Campaign

Request volume is one of the signals in OpenAI’s account of a distillation campaign. In our report, OpenAI says high-volume spikes on 24 and 25 July consisted of 16,000 requests using a relevant extraction pattern from over 4,000 users, and that related prompt-pattern activity across a cluster of more than 15,000 users was fully disrupted by 28 July.

From our news coverage

The post is OpenAI’s own account, dated 30 September 2026, and our report verified none of it independently.

The Business Engineer Lens

Where this idea sits in the Business Engineer library:

Business Engineer Framework

The Map of AI

The Business Engineer framework maps where each part of the AI stack sits and who holds the leverage. These pills are short lessons from the same library.

Explore the Map of AI →

The Question to Ask

  1. What happens to our product in its busiest minute?

How This Concept Connects

Layer 3 of 8 · Inference & Context, on the Business Engineer Concept Map

GPUsWhy AI Canโ€™t Be Both InstaDeclared IdentityRate Limits
  • Rate Limits builds on GPUs. Providers cap requests because their chips are the kitchen: computing power is shared and finite.
  • Rate Limits contrasts with Why AI Canโ€™t Be Both Instant and Cheap. Both are about busy capacity: rate limits cap requests per minute, batching trades each user’s wait for more answers overall.
  • Rate Limits builds on Declared Identity (lesson unlocks soon). A cap on how many requests a customer may send needs to know which customer is asking, which declared identity provides.
Business Engineer framework
The State of the Inference Economy
Shared, finite serving capacity.
See all 141 concepts on one map
Get the full Concept Map as a free poster.
Get the map

Related Business Pills

Business PillWhy AI Canโ€™t Be Both Instant and Cheap: Latency vs ThroughputWhy AI canโ€™t be both instant and cheap: latency versus throughput, batching, and why the same model is sold at different prices.Business PillUsage-Based Pricing Explained: Paying for Work, Not for SeatsWhat usage-based pricing is and why AI agents push vendors toward it: a seat-based example that halves a vendorโ€™s revenue, per seat versus per use, and the one question to ask.Business PillPrompt Caching Explained: Why You Should Not Pay Twice to Read the Same PageWhat prompt caching is in AI: storing the part of a request that repeats so the model does not process it again, the two catches, and the one question to ask.Business PillModel Routing Explained: Sending Each Question to the Right AI ModelWhat model routing is: sending each request to the cheapest model that can handle it, easy questions to a small model and hard ones to a big one, the risk of a wrong turn, and the one question to ask.

All Business Pills

Scroll to Top

Discover more from FourWeekMBA

Subscribe now to keep reading and get access to the full archive.

Continue reading

FourWeekMBA