What generative AI is: models that create text, images, audio and code
Generative AI is a type of artificial intelligence that can create new content such as text, images, audio, and code in response to a prompt. It works by training deep learning models on large amounts of data to learn patterns and relationships, then using that knowledge to generate novel outputs. These models are built and refined through phases of training, tuning, and evaluation.
Definition and scope
Generative AI, sometimes called gen AI, is a branch of artificial intelligence that produces new material like text, pictures, video, sound, or software code when a user gives a prompt, as explained in IBM's generative AI overview. It is a subset of AI that focuses on generating new data rather than making predictions about existing data.
According to AWS's explanation of generative AI, this technology can learn languages, programming, art, science, and other complex subjects, and it applies what it has learned to tackle new problems. For instance, it can take English vocabulary and craft a poem from those words.
The MIT News article on generative AI explains that before the recent boom, AI typically referred to machine-learning models that make predictions based on data, such as predicting whether an X-ray shows a tumor. Generative AI, in contrast, is trained to create new data that looks like the data it was trained on.
How generative AI works
Generative AI relies on deep learning models that identify and encode patterns and relationships in huge amounts of data, as noted in IBM's generative AI overview. The training process involves feeding a deep learning algorithm vast volumes of raw, unstructured data, and the algorithm performs millions of fill-in-the-blank exercises to predict the next element in a sequence, adjusting itself to minimize errors.
The NVIDIA glossary entry on generative AI explains that generative AI models use neural networks to identify patterns and structures within existing data. One breakthrough is the use of unsupervised and semi-supervised learning, which allows organizations to leverage large amounts of unlabeled data to create foundation models.
The Microsoft Learn module on generative AI describes large language models (LLMs) as a core component, and emphasizes the importance of prompts in guiding the model's output. The module also introduces AI agents, which use generative AI to perform tasks.
Training, tuning, and generation
According to IBM's generative AI overview, generative AI operates in three phases: training to create a foundation model, tuning to tailor the model to a specific application, and generation, evaluation, and retuning to improve output quality. Training is compute-intensive and requires thousands of graphics processing units (GPUs) and weeks of processing.
Tuning methods include fine-tuning, where the model is fed labeled data specific to the application, and reinforcement learning with human feedback (RLHF), where human users evaluate generated content to help the model improve, as described in IBM's generative AI overview. Retrieval augmented generation (RAG) is another technique that extends the model to use relevant sources outside the training data.
The NVIDIA glossary entry on generative AI describes several types of generative models, including diffusion models, variational autoencoders (VAEs), and generative adversarial networks (GANs). Diffusion models add random noise to training data and then reverse the process to generate new data, while GANs pit a generator against a discriminator to produce realistic outputs.
Applications and evaluation
Generative AI has many use cases across industries. AWS's explanation of generative AI lists examples in financial services, healthcare, automotive, media, and telecommunications. For instance, financial institutions use chatbots to generate product recommendations, and healthcare companies use generative AI to design synthetic gene sequences or create synthetic patient data.
The NVIDIA glossary entry on generative AI states that the three key requirements of a successful generative AI model are quality, diversity, and speed. Quality is important for user-facing applications, diversity helps reduce biases, and speed is needed for interactive applications like real-time image editing.
The NIST publication on the AI Risk Management Framework provides a cross-sectoral profile for managing risks associated with generative AI. It is a companion resource to the AI RMF 1.0, intended for voluntary use to improve the ability of organizations to incorporate trustworthiness into the design, development, and evaluation of AI systems.
| Model type | How it works | Strengths | Weaknesses |
|---|---|---|---|
| Diffusion models | Adds noise to data, then reverses to generate new data | High-quality output, flexible | Slow generation |
| Variational autoencoders (VAEs) | Encoder compresses data, decoder reconstructs it | Fast generation | Less detailed output |
| Generative adversarial networks (GANs) | Generator and discriminator compete | High-quality samples, fast | Weak sample diversity |
Frequently asked questions
What is generative AI?
Generative AI is a type of artificial intelligence that can create new content such as text, images, audio, and code in response to a prompt, by learning patterns from large amounts of data.
How does generative AI differ from traditional AI?
Traditional AI models make predictions based on data, such as classifying an image. Generative AI models are trained to create new data that resembles the training data, producing original content.
What are foundation models?
Foundation models are large generative AI models trained on broad data, capable of performing a wide variety of tasks. They serve as a base for applications that can be adapted to specific uses.
What are some common types of generative models?
Common types include diffusion models, variational autoencoders (VAEs), and generative adversarial networks (GANs). Each has different strengths in terms of output quality, speed, and diversity.
How is generative AI evaluated?
Generative AI models are evaluated on quality, diversity, and speed. Quality ensures outputs are useful, diversity reduces bias, and speed is important for interactive applications.