"How does a computer create a photorealistic face of someone who never existed? Three big ideas — and today, diffusion leads."
Level: Intermediate · Time: ~5 days · Prerequisites: basic neural networks
Learning Objectives
By the end of this module, you will be able to:
- Explain what a generative model does
- Compare GANs, VAEs, and diffusion at a high level
- Understand why diffusion models dominate image generation
- Recognize the trade-offs among the three
1. What Is a Generative Model?
A generative model learns the patterns of a dataset well enough to create brand-new examples that look like they belong. Trained on millions of faces, it can produce a new, realistic face. Trained on music, new music.
The key word is distribution — the "shape" of all the data. Imagine plotting every real face as a point in a vast space; they cluster into a specific region (two eyes above a nose above a mouth, plausible skin tones, and so on). A classifier just draws boundaries in that space ("cat region vs dog region"). A generative model does something harder: it learns the region's whole shape, then samples a fresh point from inside it — a face that's new but still lands in "plausible-face" territory rather than random noise.
Concept: Classifiers answer "what is this?" Generative models answer "what could a new one look like?" They learn the underlying distribution of the data, then sample fresh points from it.
2. GANs — The Forger vs the Detective
A GAN pits two networks against each other: a generator that forges fakes and a discriminator that tries to spot them. As each improves, the forgeries get more convincing.
Explain like I'm new: Picture an art forger and a detective. The forger keeps painting fakes; the detective keeps catching them. Round after round, the forger gets so good the detective can't tell — and now you have realistic generated images.
GANs produce sharp images but can be unstable to train and sometimes ignore parts of the data ("mode collapse").
3. VAEs — Compress, Then Recreate
A Variational Autoencoder squeezes data into a compact "latent" summary and learns to rebuild it. Sample a new point in that latent space, decode it, and you get a new example.
VAEs are stable and give a smooth, organized latent space (great for exploring variations), but their outputs tend to be blurrier than GANs'.
4. Diffusion — Add Noise, Then Reverse It
Diffusion models learn to slowly turn a real image into pure static, then reverse the process — starting from static and denoising step by step into a fresh image.
In other words, it learns to reverse noise into a clean image, one small step at a time.
Explain like I'm new: Picture a photo slowly dissolving into TV static over 50 steps. Diffusion trains a model to undo one step of that dissolving — to make a slightly-less-noisy version of any noisy image. To generate something new, you start from pure static and apply that "denoise one step" model 50 times. Static in, a fresh photo out. Breaking the impossible task ("make an image from nothing") into 50 tiny, learnable nudges is the whole trick.
Concept: It sounds odd — "learn to undo noise" — but it's remarkably stable and produces stunning quality. This is the technology behind modern image generators, and it now extends to video and audio.
| Model | Strength | Weakness |
|---|---|---|
| GAN | Sharp, fast to sample | Unstable training, mode collapse |
| VAE | Stable, smooth latent space | Blurrier outputs |
| Diffusion | Highest quality, stable | Slower to sample (many steps) |
5. Why Diffusion Won (For Now)
Diffusion combines stable training with top-tier quality — the best of both older approaches — which is why it powers today's leading image tools. Research now focuses on making it faster (fewer denoising steps).
The speed problem is the obvious downside: if GANs generate an image in one pass and diffusion needs 50, diffusion is slower by design. A major research thread is "distilling" diffusion so it produces comparable quality in just a handful of steps — some recent methods get close to a single step. When you hear about an image generator becoming "10x faster," it's usually this: squeezing the many-step denoising process into far fewer steps without losing quality.
Common mistake: Assuming one approach is universally "best." Each has trade-offs; GANs are still great when sampling speed matters, VAEs when you want a tidy latent space. Diffusion leads on quality, not on every axis.
✅ Checkpoint
- What does a generative model learn to do?
- How does a GAN's "forger vs detective" setup work?
- What is the core trick behind diffusion models?
Answers: 1) Learn a dataset's patterns well enough to create new, realistic examples. 2) A generator makes fakes and a discriminator judges them; competition drives both to improve. 3) Learn to reverse a noising process — start from static and denoise step by step into a new image.
Key Takeaway: Generative models create new data that fits a learned distribution. GANs (forger vs detective) are sharp but unstable; VAEs (compress then recreate) are stable but blurrier; diffusion (add noise, then learn to reverse it) combines stability with the highest quality, making it the current leader for images — with speed as the active research frontier.
Further Learning
Part of "Research & Advanced AI." Original content for this learning platform.