Before anyone teaches it a task, a model plays a simple game. Take a sentence, hide the next word, and ask the model to guess it. Then show the real word and adjust the model a little, so its next guess is better.
Trillions of Guesses
Now repeat that game trillions of times, across books, websites, code and conversations. Nobody labels anything. The text itself provides the answer. That is what makes it possible at this scale.
To guess well, the model has to pick up a great deal along the way: grammar, facts, how an argument is built, how code works. None of it is taught directly. It all comes from predicting the next word.
The Expensive Stage
This stage is called pretraining. It produces a general model, one that knows a lot but has no particular job yet. It is also the expensive stage: thousands of chips running for months. Very few organizations can afford it.
But it is done once, and the result can be reused for thousands of different purposes. This shapes the whole industry: a handful of companies build the general models, and everyone else builds on top of them.
Why It Matters
A model only knows what was in its training text, up to the day that text was collected.
Pretraining builds on Parameters. Pretraining nudges the model a little after every guess, and those adjustments are the parameters the dials lesson describes.
Pretraining contrasts with Data Labeling. Pretraining needs nobody to label anything because the text supplies the answer; data labeling needs people to attach answers.
Knowledge Cutoff builds on Pretraining. A model knows only its training text up to the day it was collected, which the cutoff lesson names as the knowledge cutoff.
Fine-Tuning builds on Pretraining. Fine-tuning is extra training on your own examples, applied on top of the general model pretraining produces.
RLHF builds on Pretraining. RLHF starts from a model that learned to continue sentences by reading enormous text, then trains it toward helpful answers.
Inference contrasts with Pretraining. Pretraining is the expensive stage paid once; inference is paid on every answer, small and forever.
Temperature builds on Pretraining. Temperature sets how much chance the model allows when picking the next word, the guessing game pretraining teaches.
Physical AI(lesson unlocks soon) builds on Pretraining. The new robots learn the way chat models did, from enormous numbers of examples, rather than following a fixed script.
The Business Engineer framework maps where each part of the AI stack sits and who holds the leverage. These pills are short lessons from the same library.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
Scroll to Top
Discover more from FourWeekMBA
Subscribe now to keep reading and get access to the full archive.
