Variational Autoencoder (VAE)
A Variational Autoencoder (VAE) is a generative neural network architecture, introduced by Diederik Kingma and Max Welling in 2013, that learns to compress data into a probabilistic, continuous latent representation and then reconstruct or…
Definition
A Variational Autoencoder (VAE) is a generative neural network architecture, introduced by Diederik Kingma and Max Welling in 2013, that learns to compress data into a probabilistic, continuous latent representation and then reconstruct or generate new data from that representation, combining ideas from autoencoders and Bayesian probabilistic modeling.
Overview
A standard autoencoder consists of an encoder that compresses input data into a compact latent representation and a decoder that reconstructs the original input from that representation, trained to minimize reconstruction error. A VAE modifies this setup in a key way: instead of encoding an input into a single fixed point in latent space, the encoder outputs the parameters (mean and variance) of a probability distribution, and a latent vector is then sampled from that distribution before being passed to the decoder. This makes the latent space continuous and well-structured, meaning that points sampled anywhere in the space tend to decode into plausible, coherent outputs. VAEs are trained with a loss function combining two terms: a reconstruction loss (how well the decoder recreates the input) and a regularization term (KL divergence) that encourages the learned latent distributions to stay close to a simple prior, typically a standard normal distribution. This regularization is what gives VAEs their key generative property — because the latent space is organized and continuous rather than an arbitrary set of disconnected points, new data can be generated by sampling from the prior distribution directly and passing that sample through the decoder, without needing a corresponding real input. Compared to GANs, VAEs are generally easier and more stable to train, and they provide an explicit, interpretable latent space useful for tasks like interpolation between data points or controlled attribute editing, but their generated outputs have historically tended to look somewhat blurrier or less sharp than high-quality GAN or diffusion model outputs. In modern generative AI systems, VAEs are frequently used as a component rather than a standalone generator: many text-to-image diffusion models, including Stable Diffusion, use a VAE to compress images into a smaller latent space before running the diffusion process there (rather than on full-resolution pixels), which substantially reduces the computational cost of generation.
Key Concepts
- Encodes input data into a probabilistic latent distribution rather than a fixed point
- Combines reconstruction loss with a KL-divergence regularization term
- Produces a continuous, well-structured latent space suitable for sampling new data
- Generally more stable and easier to train than GANs
- Supports latent-space interpolation and controlled attribute editing
- Used as a compression component inside latent diffusion models like Stable Diffusion