Model routing means sending each request to the cheapest model that can handle it. Easy questions go to a small model and hard ones to a big model, so the bill drops without the quality dropping.
A short explainer video, under a minute. The customer and the robots are an illustration.
At a glance
What Model Routing Is
So the company adds a sorter at the door.
The Board: Easy and Hard
The video's board has two columns. Easy questions go to the small model, where the cost is low.
Why It Matters
The video turns this into one question: are we paying the top price for simple questions?
Picture a company that sends every question to its biggest, smartest robot. A customer asks what time the shop opens. The big robot thinks hard and answers correctly, but slowly and at a high price.
Most questions are that simple. A small robot could answer them faster, for a fraction of the cost.
What Model Routing Is
So the company adds a sorter at the door. That is called model routing: sending each request to the cheapest model that can handle it.
Easy questions go to the small model. Hard ones go to the big one.
The Board: Easy and Hard
The video’s board has two columns. Easy questions go to the small model, where the cost is low. Hard questions go to the big model, where the cost is high, but rare.
The result in the video: the bill drops, without the quality dropping.
The Idea on One Board
The Board: Easy and Hard
Easy questions
Easy questions go to the small model, where the cost is low.
Hard questions
Hard questions go to the big model, where the cost is high, but rare.
The Risk Is a Wrong Turn
The risk is a wrong turn. A hard question sent to the small robot gets a weak answer.
That is why the routes need checking.
Why It Matters
The video turns this into one question: are we paying the top price for simple questions?
Not every question needs the biggest model. Send each one to the cheapest that does it well.
The key insight: Not every question needs the biggest model. Send each one to the cheapest that does it well.
The honest limit
The risk is a wrong turn. A hard question sent to the small robot gets a weak answer.
Our report on Ai2’s AstaBrief says that, according to Ai2, Asta now offers a Fast mode on its open model alongside a Claude-powered Thinking mode, and that 23% of users who tried Fast mode never switched back to Thinking mode while another 18% moved between the two. In that product the user picks the route; this lesson is about a system picking it.
The Business Engineer Lens
Where this idea sits in the Business Engineer library:
The Business Engineer framework maps where each part of the AI stack sits and who holds the leverage. These pills are short lessons from the same library.
Mixture of Experts contrasts with Model Routing. Model routing picks a whole model per request; mixture of experts routes each question to a few specialists inside one model.
Reasoning Models contrasts with Model Routing. Both match effort to difficulty: reasoning spends more thinking on hard tasks, routing sends easy ones to a cheaper model.
Model Routing builds on Parameters. Routing rests on the point that bigger models cost more per answer and a smaller one often does the job.
Model Routing drives the cost of Gross Margin. Sending simple questions to a small model cuts the cost of each answer, the direct cost gross margin subtracts.
Prompt Caching(lesson unlocks soon) contrasts with Model Routing. Routing cuts cost by choosing a cheaper model; caching cuts it by not reprocessing the unchanged part of each request.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
Scroll to Top
Discover more from FourWeekMBA
Subscribe now to keep reading and get access to the full archive.
