A lab spends months and a fortune teaching a robot. Then the teaching stops. The robot goes to work, and every single answer uses chips and electricity.
That second stage has a name. It is called inference.
What Inference Is
Inference is running a trained model to produce an answer.
Training is the first stage, where the model is taught. Inference is the second, where it is put to work.
Two Different Bills
Training is paid once, large and upfront. Inference is paid on every answer, small and forever.
With a few customers, the small bill stays small. With millions, it passes the big one, and it keeps growing.
The Idea on One Board
Two Different Bills
Training
Training is paid once, large and upfront.
Inference
Inference is paid on every answer, small and forever.
Why It Matters
The cost of an answer is small, but the number of answers is not fixed. The total depends on how many answers are asked for.
The video’s question is what each answer costs, and how many answers are needed.
The key insight: The cost of an answer is small, but the number of answers is not fixed. The total depends on how many answers are asked for.
The honest limit
With a few customers, the small bill stays small. With millions, it passes the big one, and it keeps growing.
Our report on Ai2’s AstaBrief says that, according to Ai2, the Fast mode in its Asta platform averages 51.1 seconds per report against 178.5 seconds for the Claude-powered Thinking mode, which Ai2 calls about 3.5 times faster, and that Ai2 says most of its evaluation was done in 2025. Running a model to produce each answer is what this lesson calls inference.
The Business Engineer Lens
Where this idea sits in the Business Engineer library:
The Business Engineer framework maps where each part of the AI stack sits and who holds the leverage. These pills are short lessons from the same library.
The Memory Wall constrains Inference. Because each word needs one read of the model, memory bandwidth caps how fast inference can produce answers.
Parameters drives the cost of Inference. More parameters cost more on every single answer, not just once, which is the per-answer bill of inference.
Mixture of Experts drives the cost of Inference. Only a few experts work on each question, which cuts the work per answer even though every expert still takes space.
Multimodal drives the cost of Inference. A picture costs more to process than a sentence, so taking in images and sound raises the cost of each answer.
Inference is priced by Tokens. Every answer uses chips and electricity, and models are priced in tokens, the unit the AI economy bills in.
Inference drives the cost of Gross Margin. Inference is paid on every answer, so serving the same customer costs an AI company far more than a software company.
Inference contrasts with Pretraining. Pretraining is the expensive stage paid once; inference is paid on every answer, small and forever.
Inference constrains Operating Leverage. Because every AI answer has a cost, costs rise with use, which weakens the operating leverage of build-once software.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
Scroll to Top
Discover more from FourWeekMBA
Subscribe now to keep reading and get access to the full archive.
