Generative models are those that equip computers with a better understanding of the world experienced by humans.
Understanding generative models
Most of us take our understanding of the physical world for granted, while others may have never stopped to think about how much they know.
The three-dimensional world we inhabit is made up of objects that move and collide. Animals that fly, swim, bark, and quack. People that interact, discuss, think, and walk. Computer monitors that display information about how to prune a bonsai, who won a football match, or what happened in the year 1975.
Most of the information we are exposed to is accessible to us in either physical or digital form. But this is not the case for machine learning models and the algorithms on which they are based. To create AI that can analyze and then understand the diverse human experience, generative models may be the answer.
The emergence of generative AI
In its 2022 Emerging Technologies and Trends Impact Radar, Gartner listed generative AI as one of five rapidly evolving technologies that will play a part in the productivity revolution.
Some of Gartnerโs key predictions include:
- By 2025 โ generative AI will produce 10% of all data and 20% of all test data related to consumer-facing use cases. It will also be incorporated into 50% of all drug discovery and development ventures.
- By 2027 โ generative AI will be used by 30% of all manufacturers to increase the effectiveness of product development.
Gartner noted that generative AI methods were proving themselves in a wide range of industries such as life sciences, healthcare, automotive, aerospace, material science, media, entertainment, defense, and energy.
How are generative models trained?
Generative AI algorithms undergo unsupervised and semi-supervised learning that enables them to create new content from existing content like text, audio, video, and even code. The overarching objective of a generative model is to create original content that is also plausible.
To train these models, vast amounts of data are first sourced from a particular domain such as sounds or images. Then it is a matter of training the model to produce similar content.
The neural networks OpenAI uses as generative models, for example, contain several parameters that are much smaller than the amount of data it uses to train them. According to the company, this forces the model to โdiscover and efficiently internalize the essence of the data in order to generate it.โ
The GAN approach
OpenAI uses the example of a network that it wants to train to generate 200 realistic images. To ensure the images look real, the company employs what it calls the Generative Adversarial Network (GAN) approach.
The approach involves the introduction of another standard neural network that serves as a discriminator and tries to classify whether an input image is real or fake. OpenAI admitted that it could serve the model with 200 real images and 200 generated images and ask it to train a standard classifier.
But a better strategy was to change the parameters of the generative AI model to make the 200 samples more confusing to the discriminator. This would result in a battle between the two networks: the discriminator wants to tell the difference between real and generated images, while the generator wants to produce images that make the discriminator believe they are real.
Ultimately, the generative model wins because, from the discriminatorโs point of view, it produces images that are indistinguishable from the real thing.
OpenAIโs model was ultimately forced to compress 200GB of pixel data into just 100MB of weights which encouraged it to identify the most important features of the data. In the context of the modelโs training to create realistic images from scratch, it learned that:
- Pixels in close proximity are more likely to be the same color.
- The world is comprised of horizontal and vertical edges and blobs of solid color.
- Certain objects, textures, and backgrounds occur in certain arrangements and, in video, transform over time in specific ways.
Current and future applications of generative models
Generative models have many short-term applications such as structured prediction, image denoising, super-resolution imaging, and also in pre-training where access to labeled data is prohibitively expensive.
As generative models are trained over the long term, however, it is hoped the AI will develop a fundamental understanding of the world and the elements with which it is comprised. With access to data once off-limits to technology, it is likely AI will become an increasingly powerful and versatile force for consumers and businesses alike.
Key takeaways
- Generative models are those that equip computers with a better understanding of the world experienced by humans.
- Gartner listed generative AI as one of five rapidly evolving technologies that will play a part in the productivity revolution. Generative models are already effective in life sciences, healthcare, automotive, aerospace, material science, media, entertainment, and defense and energy.
- Generative AI algorithms undergo unsupervised and semi-supervised learning that enables them to create new content from existing content like text, audio, video, and even code. The overarching objective of a generative model is to create original content that is also plausible.
Connected AI Concepts

Deep Learning vs. Machine Learning




OpenAI Organizational Structure




Stability AI Ecosystem

Main Free Guides: