Generative models are those that equip computers with a better understanding of the world experienced by humans.
Understanding generative models
Most of us take our understanding of the physical world for granted, while others may have never stopped to think about how much they know.
The three-dimensional world we inhabit is made up of objects that move and collide. Animals that fly, swim, bark, and quack. People that interact, discuss, think, and walk. Computer monitors that display information about how to prune a bonsai, who won a football match, or what happened in the year 1975.
Most of the information we are exposed to is accessible to us in either physical or digital form. But this is not the case for machine learning models and the algorithms on which they are based. To create AI that can analyze and then understand the diverse human experience, generative models may be the answer.
The emergence of generative AI
In its 2022 Emerging Technologies and Trends Impact Radar, Gartner listed generative AI as one of five rapidly evolving technologies that will play a part in the productivity revolution.
Some of Gartner’s key predictions include:
By 2025 – generative AI will produce 10% of all data and 20% of all test data related to consumer-facing use cases. It will also be incorporated into 50% of all drug discovery and development ventures.
Gartner noted that generative AI methods were proving themselves in a wide range of industries such as life sciences, healthcare, automotive, aerospace, material science, media, entertainment, defense, and energy.
How are generative models trained?
Generative AI algorithms undergo unsupervised and semi-supervised learning that enables them to create new content from existing content like text, audio, video, and even code. The overarching objective of a generative model is to create original content that is also plausible.
To train these models, vast amounts of data are first sourced from a particular domain such as sounds or images. Then it is a matter of training the model to produce similar content.
The neural networks OpenAI uses as generative models, for example, contain several parameters that are much smaller than the amount of data it uses to train them. According to the company, this forces the model to “discover and efficiently internalize the essence of the data in order to generate it.”
The GAN approach
OpenAI uses the example of a network that it wants to train to generate 200 realistic images. To ensure the images look real, the company employs what it calls the Generative Adversarial Network (GAN) approach.
The approach involves the introduction of another standard neural network that serves as a discriminator and tries to classify whether an input image is real or fake. OpenAI admitted that it could serve the model with 200 real images and 200 generated images and ask it to train a standard classifier.
But a better strategy was to change the parameters of the generative AI model to make the 200 samples more confusing to the discriminator. This would result in a battle between the two networks: the discriminator wants to tell the difference between real and generated images, while the generator wants to produce images that make the discriminator believe they are real.
Ultimately, the generative model wins because, from the discriminator’s point of view, it produces images that are indistinguishable from the real thing.
OpenAI’s model was ultimately forced to compress 200GB of pixel data into just 100MB of weights which encouraged it to identify the most important features of the data. In the context of the model’s training to create realistic images from scratch, it learned that:
Pixels in close proximity are more likely to be the same color.
The world is comprised of horizontal and vertical edges and blobs of solid color.
Certain objects, textures, and backgrounds occur in certain arrangements and, in video, transform over time in specific ways.
Current and future applications of generative models
Generative models have many short-term applications such as structured prediction, image denoising, super-resolution imaging, and also in pre-training where access to labeled data is prohibitively expensive.
As generative models are trained over the long term, however, it is hoped the AI will develop a fundamental understanding of the world and the elements with which it is comprised. With access to data once off-limits to technology, it is likely AI will become an increasingly powerful and versatile force for consumers and businesses alike.
Key takeaways
Generative models are those that equip computers with a better understanding of the world experienced by humans.
Gartner listed generative AI as one of five rapidly evolving technologies that will play a part in the productivity revolution. Generative models are already effective in life sciences, healthcare, automotive, aerospace, material science, media, entertainment, and defense and energy.
Generative AI algorithms undergo unsupervised and semi-supervised learning that enables them to create new content from existing content like text, audio, video, and even code. The overarching objective of a generative model is to create original content that is also plausible.
Generalized AI consists of devices or systems that can handle all sorts of tasks on their own. The extension of generalized AI eventually led to the development of Machine learning. As an extension to AI, Machine Learning (ML) analyzes a series of computer algorithms to create a program that automates actions. Without explicitly programming actions, systems can learn and improve the overall experience. It explores large sets of data to find common patterns and formulate analytical models through learning.
Machine learning is a subset of artificial intelligence where algorithms parse data, learn from experience, and make better decisions in the future. Deep learning is a subset of machine learning where numerous algorithms are structured into layers to create artificial neural networks (ANNs). These networks can solve complex problems and allow the machine to train itself to perform a task.
DevOps refers to a series of practices performed to perform automated software development processes. It is a conjugation of the term “development” and “operations” to emphasize how functions integrate across IT teams. DevOps strategies promote seamless building, testing, and deployment of products. It aims to bridge a gap between development and operations teams to streamline the development altogether.
AIOps is the application of artificial intelligence to IT operations. It has become particularly useful for modern IT management in hybridized, distributed, and dynamic environments. AIOps has become a key operational component of modern digital-based organizations, built around software and algorithms.
Machine Learning Ops (MLOps) describes a suite of best practices that successfully help a business run artificial intelligence. It consists of the skills, workflows, and processes to create, run, and maintain machine learning models to help various operational processes within organizations.
OpenAI is an artificial intelligence research laboratory that transitioned into a for-profit organization in 2019. The corporate structure is organized around two entities: OpenAI, Inc., which is a single-member Delaware LLC controlled by OpenAI non-profit, And OpenAI LP, which is a capped, for-profit organization. The OpenAI LP is governed by the board of OpenAI, Inc (the foundation), which acts as a General Partner. At the same time, Limited Partners comprise employees of the LP, some of the board members, and other investors like Reid Hoffman’s charitable foundation, Khosla Ventures, and Microsoft, the leading investor in the LP.
OpenAI has built the foundational layer of the AI industry. With large generative models like GPT-3 and DALL-E, OpenAI offers API access to businesses that want to develop applications on top of its foundational models while being able to plug these models into their products and customize these models with proprietary data and additional AI features. On the other hand, OpenAI also released ChatGPT, developing around a freemium model. Microsoft also commercializes opener products through its commercial partnership.
OpenAI and Microsoft partnered up from a commercial standpoint. The history of the partnership started in 2016 and consolidated in 2019, with Microsoft investing a billion dollars into the partnership. It’s now taking a leap forward, with Microsoft in talks to put $10 billion into this partnership. Microsoft, through OpenAI, is developing its Azure AI Supercomputer while enhancing its Azure Enterprise Platform and integrating OpenAI’s models into its business and consumer products (GitHub, Office, Bing).
Stability AI is the entity behind Stable Diffusion. Stability makes money from our AI products and from providing AI consulting services to businesses. Stability AI monetizes Stable Diffusion via DreamStudio’s APIs. While it also releases it open-source for anyone to download and use. Stability AI also makes money via enterprise services, where its core development team offers the chance to enterprise customers to service, scale, and customize Stable Diffusion or other large generative models to their needs.
Gennaro is the creator of FourWeekMBA, which reached about four million business people, comprising C-level executives, investors, analysts, product managers, and aspiring digital entrepreneurs in 2022 alone | He is also Director of Sales for a high-tech scaleup in the AI Industry | In 2012, Gennaro earned an International MBA with emphasis on Corporate Finance and Business Strategy.
