{
"cells": [
{
"cell_type": "markdown",
"id": "797ac476",
"metadata": {},
"source": [
"# Diffusion, Score, and Transport-Based Generation\n",
"\n",
"*The Mathematics of Large Language Models ยท Chapter 11*\n",
"\n",
"Track noise, read scores and compare two reverse journeys.\n",
"\n",
"These pages illustrate the mathematics. Read the saved figures and calculations in order, or open [the interactive browser edition](reader.html) to change the examples. No code editing is needed.\n",
"\n",
"Request the observed measurement, noise or blur model, available conditioning and the intended interpretation. Use a toy posterior with multiple compatible originals to explain uncertainty; return compatible alternatives and identify which details are supported by the observation versus supplied by a prior. For numeric diffusion calculations request alpha bar, the clean-data distribution and noise variance. Distinguish conditional score given a clean original from the marginal score. Explain supplied guidance settings through a clearly labeled toy score blend; consult product documentation before describing unsupplied current settings."
]
},
{
"cell_type": "markdown",
"id": "1c8f7e5a",
"metadata": {},
"source": [
"Reproducing these calculations
\n",
"\n",
"The optional calculation cells use Python with NumPy, SciPy, and Matplotlib. All inputs are included in this file. The figures below are saved, so running these cells is optional.\n",
"\n",
"Show the answer
\n",
"\n",
"Signal variance is 0.5 x 4 = 2 and noise variance is 0.5, so the ratio is 4. The unit-variance formula alpha bar / (1 - alpha bar) would give 1.\n",
"\n",
"Show the answer
\n",
"\n",
"Zero, because the two shares are equal and the two component scores cancel. Zero alone does not say whether the center is a maximum or a minimum.\n",
"\n",
"Show the answer
\n",
"\n",
"The score is -2/(1+1) = -1. With f = 0 and g = 1 the SDE coefficient is 1 and the ODE coefficient is 0.5. Since dt is negative, x moves toward 0 in both cases.\n",
"\n",
"Show the answer
\n",
"\n",
"No. Conditioning can favor an alternative the prior likes, but it cannot create evidence that separates originals the measurement does not distinguish.\n",
"\n",
"Show the answer
\n",
"\n",
"Below, at 1.97. The two weights 0.513 and 0.472 add to 0.986, less than 1, so the mean is pulled slightly toward 0.\n",
"\n",
"Show the answer
\n",
"\n",
"No, it grows about seven times, from 0.009 to 0.060. The far/near ratio is exp(-4/epsilon) (so near/far is exp(4/epsilon)), which changes quickly when epsilon is small.\n",
"\n",
"Show the answer
\n",
"\n",
"About the same: the time covered is 24 against 20, and what matters for crossing is time, not step count. Only about 6% of chains cross at h = 0.1 here.\n",
"\n",
"Show the answer
\n",
"\n",
"0.25, which is 1/4. For the linear schedule the chance is 1/t whatever T is.\n",
"\n",
"