Capstone: Autoencoders, GANs & Diffusion

Discriminating vs. generating

Everything so far discriminates: photo in → "cat" out. Generative models run the arrow the other way: learn the distribution of the data (probability path!) well enough to draw new samples from it — new faces, new sentences, new molecules. Language models are generative (sampling the next token); here we focus on images, where three ideas built today's tools.

Autoencoders: creation via compression

An autoencoder is trained on the humblest objective: reproduce the input. The catch — the network must squeeze through a narrow bottleneck:

image (millions of dims) → ENCODER → latent vector z (e.g. 128 dims) → DECODER → image

To reconstruct through 128 numbers, the network is forced to discover the essence of its domain — for faces: pose, lighting, expression, identity. That latent space is a learned, compact coordinate system for the data (embeddings again!). Variational autoencoders (VAEs) regularize this space to be smooth, so any latent point decodes to a plausible image — walk through latent space and one face morphs continuously into another. VAE samples tend to be blurry, but the latent-space idea became foundational.

GANs: the forger and the detective

Generative Adversarial Networks (2014) train two networks against each other:

The generator turns random noise into fake images.
The discriminator tries to tell fakes from real training images.

The generator improves precisely to fool the discriminator; the discriminator sharpens in response — an arms race whose gradients flow from the detective back into the forger. At equilibrium, fakes are indistinguishable from data. GANs produced the first photorealistic synthetic faces (StyleGAN — thispersondoesnotexist.com) and the term "deepfake". Their curse: adversarial training is notoriously unstable — the pursuit of something steadier led directly to…

Diffusion: sculpting images out of noise

The idea behind DALL·E, Stable Diffusion and Midjourney is disarmingly simple:

Forward (fixed): gradually add Gaussian noise to a training image over ~1,000 steps until pure static remains.
Learn the reverse: train a network to predict-and-remove a little noise at each step.
Generate: start from pure random noise and repeatedly denoise — an image crystallizes.

Each denoising step is an ordinary supervised regression (predict the added noise — MSE loss!), so training is stable, no arms race. Text conditioning feeds a prompt's embedding (from a language model) into every denoising step, steering what crystallizes; latent diffusion (Stable Diffusion) runs the whole process inside a VAE's compact latent space to make it affordable. Slower to sample than GANs — many steps — but quality and diversity won: diffusion is today's image-generation standard.

You've completed Deep Learning

The arc: neurons → layers → backprop → CNNs → RNNs → transformers → generative models. You now understand the machinery inside every headline AI system. The AI path zooms out: agents, reasoning, LLMs as systems, ethics — the big picture built atop these components.

Key takeaways

Autoencoders learn compressed latent essences; VAEs make latent space smooth and sampleable.
GANs: generator vs. discriminator — brilliant, photorealistic, unstable.
Diffusion: learn to denoise step-by-step; stable training, text-steerable — the engine of modern image generators.