Diffusion Models Cheat Sheet

Covers the forward noising process, learned reverse denoising process, and practical text-to-image inference with the Hugging Face diffusers library.

2 PagesAdvancedMar 2, 2026

Core Concepts

How diffusion models generate data.

Forward (noising) process- Fixed process that gradually adds Gaussian noise to data over T timesteps until it becomes pure noise
Reverse (denoising) process- A neural network learns to predict and remove noise at each timestep, step by step turning noise back into data
Noise schedule (beta_t)- Controls how much noise is added at each timestep; linear and cosine schedules are common
Denoising objective- The model (typically a U-Net) is trained to predict the noise epsilon added at a given timestep, minimizing MSE
Classifier-free guidance- Blends conditional and unconditional noise predictions at inference to strengthen adherence to a text prompt
Latent diffusion- Runs the diffusion process in a compressed VAE latent space instead of pixel space for efficiency (e.g., Stable Diffusion)

Text-to-Image Inference with Diffusers

Generate an image from a text prompt using a pretrained pipeline.

python

from diffusers import StableDiffusionPipelineimport torchpipe = StableDiffusionPipeline.from_pretrained(    "runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda")image = pipe(    prompt="a watercolor painting of a mountain lake at sunrise",    num_inference_steps=30,    guidance_scale=7.5,).images[0]image.save("output.png")

DDPM Reverse Sampling Step

Simplified pseudocode for one denoising step during sampling.

python

# x_t: noisy sample at timestep t; model predicts noise epsilonpredicted_noise = model(x_t, t)alpha_t = alphas[t]alpha_bar_t = alpha_bars[t]# Estimate the mean of p(x_{t-1} | x_t)mean = (1 / alpha_t.sqrt()) * (x_t - (1 - alpha_t) / (1 - alpha_bar_t).sqrt() * predicted_noise)if t > 0:    noise = torch.randn_like(x_t)    x_t_minus_1 = mean + betas[t].sqrt() * noiseelse:    x_t_minus_1 = mean

Samplers / Schedulers

Different numerical methods to run the reverse process, trading speed for quality.

DDPM- Original formulation; typically needs hundreds to ~1000 steps for high quality
DDIM- Deterministic, non-Markovian sampler that produces good results in far fewer steps (e.g., 20-50)
Euler / Euler Ancestral- ODE-solver-based schedulers popular for fast, high-quality sampling in Stable Diffusion pipelines
DPM-Solver++- High-order solver that reaches good quality in as few as 15-25 steps

Pro Tip

Guidance scale is a quality/diversity trade-off, not a free lunch -- pushing it too high (beyond roughly 12-15) over-saturates images and reduces diversity even though it looks like it should 'improve' prompt adherence.

Was this cheat sheet helpful?

Generative Adversarial Networks (GANs) Cheat Sheet

Prompt Engineering Cheat Sheet