The maths of denoising diffusion, just enough

Diffusion models are trained by destroying data with noise and learning to undo it. This section is the four equations that say exactly that, with every symbol explained: the schedule of noise, the closed form that lets you jump to any noise level in one line, the training loss, which is plain squared error on the noise that was added, and a single step of sampling. The closed form is the piece everything else rests on, because it is why training is cheap: you never simulate a thousand noising steps to make one training example.

A schedule of small noise steps#

beta_t        small amount of noise added at step t, e.g. from 0.0001 up to 0.02 over T = 1000 steps
alpha_t       1 - beta_t, how much signal survives step t
alphabar_t    alpha_1 x alpha_2 x ... x alpha_t, how much signal survives all steps up to t

One step of the forward process is x_t = sqrt(alpha_t) * x_(t-1) + sqrt(beta_t) * z. The square roots are there so that variance is preserved: if x_(t-1) has unit variance, alpha_t of it survives, beta_t of fresh noise is added, and alpha_t + beta_t = 1, so x_t has unit variance too. That is the whole reason the schedule is written in terms of beta rather than a plain noise standard deviation, and it is why data is normalised to roughly zero mean and unit variance before any of this starts.

Skip to any step directly#

x_t = sqrt(alphabar_t) * x_0  +  sqrt(1 - alphabar_t) * noise        noise ~ Normal(0, 1)

Because adding Gaussian noise repeatedly is still Gaussian, you do not need to run t noising steps to get x_t. One line produces a noisy version of a clean image at any level. This is what makes training cheap: pick a random image, a random t, and noise it in one go.

Do not take that on trust. Running the chain 500 times on 200,000 copies of a single value and comparing with the one-line version is four lines of code, and it is the check that catches a wrong schedule before it costs you a training run.

import torch
torch.manual_seed(0)

T = 1000
betas = torch.linspace(1e-4, 0.02, T)
alphas = 1 - betas
abar = torch.cumprod(alphas, dim=0)

N, t_check, x0 = 200_000, 500, 0.8
x = torch.full((N,), x0)
for t in range(t_check):                                   # the long way
    x = alphas[t].sqrt() * x + betas[t].sqrt() * torch.randn(N)
print(f"chain, {t_check} steps : mean {x.mean():.4f}  std {x.std():.4f}")

a = abar[t_check - 1]                                      # the one-line way
y = a.sqrt() * x0 + (1 - a).sqrt() * torch.randn(N)
print(f"closed form, 1 line  : mean {y.mean():.4f}  std {y.std():.4f}")
print(f"what it should be    : mean {a.sqrt()*x0:.4f}  std {(1-a).sqrt():.4f}")
# chain, 500 steps : mean 0.2222  std 0.9617
# closed form, 1 line  : mean 0.2253  std 0.9622
# what it should be    : mean 0.2243  std 0.9599

The numbers the schedule actually produces are worth having in your head, because they are what makes the reverse process possible at all. With the linear schedule above, alphabar is 0.971 at t = 50, 0.659 at t = 200, 0.0786 at t = 500 and 0.00004 at t = 1000. So at the last step the original image is multiplied by 0.0064 — it is gone, to any precision that matters, which is exactly what lets sampling begin from a fresh draw of noise that has never seen a dataset.

The training objective#

loss = mean( (noise - model(x_t, t))^2 )

The model sees the noisy image and the step number and predicts the noise that was mixed in. It is plain regression with a squared error. If you know the noise, you know the clean image, since the equation above can be solved for x_0.

x_0 = ( x_t  -  sqrt(1 - alphabar_t) * eps )  /  sqrt(alphabar_t)

That scaling is the whole argument for the noise target. At t = 1000, sqrt(alphabar_t) is 0.0064, so recovering x_0 means dividing by 0.0064 — amplifying the model's error by a factor of 156. A network asked for x_0 directly at that step can do no better than output the dataset mean, and the squared error there swamps the loss from every step where the task was learnable. Predicting the noise keeps the target at unit scale for every t, so no part of the schedule dominates. Trained identically on the same data, noise prediction beat image prediction by about a quarter on sample quality in the 2-D experiment in the next section.

The full derivation from the variational bound puts a t-dependent weight in front of this squared error. The result that made diffusion practical is that dropping the weight entirely and using the plain mean squared error above works better in practice. So the loss you implement is simpler than the loss the theory hands you, and knowing that saves you looking for the weighting term in someone's code and concluding you have misread it.

One sampling step#

x_(t-1) = ( x_t - beta_t / sqrt(1 - alphabar_t) * model(x_t, t) ) / sqrt(alpha_t)  +  sqrt(beta_t) * z

z is fresh noise (none on the final step)

Subtract the predicted noise, scaled for this step, rescale, and add a little new noise. Repeat from pure noise down to step one. Faster samplers take bigger, smarter steps using the same trained model.

Two details in that line trip people up. The first is that the coefficient on the predicted noise is beta_t / sqrt(1 - alphabar_t) and not sqrt(1 - alphabar_t): you are removing only the noise this step added, not all the noise accumulated so far, so the subtraction is small. At t = 500 that coefficient is 0.0105, against a total accumulated noise scale of 0.96. Each step nudges. The second is the fresh noise z added back afterwards. Removing it makes the sampler deterministic and, with the DDPM formula, makes the samples worse, because the update is an approximation of a distribution rather than a point and the variance is part of what is being modelled. Deterministic sampling is a real option, but it needs the DDIM update rather than this one.

That DDIM update is worth writing down alongside, because it is what every fast sampler is a variant of. Predict the noise, convert it to an estimate of the clean image, then re-noise that estimate to the next noise level you want — which need not be t - 1. It is the freedom to choose the next level that lets a sampler run 20 steps instead of 1000.

x0_hat  = ( x_t - sqrt(1 - alphabar_t) * model(x_t, t) ) / sqrt(alphabar_t)
x_prev  = sqrt(alphabar_prev) * x0_hat  +  sqrt(1 - alphabar_prev) * model(x_t, t)

alphabar_prev is the noise level of the next step you have chosen, not necessarily t - 1

Classifier-free guidance#

guided = model(x_t, t, empty_prompt) + w * ( model(x_t, t, prompt) - model(x_t, t, empty_prompt) )

Train one model that sometimes sees the prompt and sometimes an empty one. At sampling time, take the direction the prompt pushes the prediction, and exaggerate it by a factor w. It costs two model evaluations per step.

Read the endpoints of that formula. At w = 0 the prompt term cancels and you get the unconditional model: a sample from the data distribution, ignoring what you asked for. At w = 1 the two unconditional terms cancel and you get the plain conditional model, which is what an honest sample from p(image given prompt) looks like. Every value above 1 is extrapolation past the model's own conditional prediction, which is why the usual range of 5 to 8 is not a mild setting: the samples produced there are not draws from the conditional distribution at all. They are draws from a sharpened version of it, which is exactly why they look better and exactly why diversity falls.

The name is historical and actively misleading. The original method, classifier guidance, ran a separately trained classifier on the noisy image and pushed the sample along the gradient of the class score. Classifier-free guidance gets the same effect from one model trained with the condition dropped at random, typically 10 per cent of the time, and needs no classifier. Nothing in the formula above involves a classifier; the word only records what it replaced.

Where the field went next#

Several recent image models use flow matching, a closely related formulation that learns a velocity carrying noise to data along straighter paths, which allows fewer sampling steps. The intuitions here carry over directly.

Concretely, flow matching interpolates linearly between noise and data, x_t = (1 - t) * x_0 + t * noise with t running from 0 to 1, and trains the network to predict the velocity noise - x_0 rather than the noise alone. Sampling is then integrating an ordinary differential equation, and because the paths are close to straight, a crude integrator with few steps gets close to the answer. The forward process, the regression objective and guidance all keep their shape; what changes is the interpolation and what the network is asked to output. If you have understood the four equations above you can read a flow-matching paper the same afternoon.