{
"cells": [
{
"cell_type": "markdown",
"id": "7ad77285",
"metadata": {},
"source": [
"# Latent, Adversarial, and Flow Models\n",
"\n",
"*The Mathematics of Large Language Models ยท Chapter 10*\n",
"\n",
"Integrate hidden variation, inspect bounds and measure missing modes.\n",
"\n",
"These pages illustrate the mathematics. Read the saved figures and calculations in order, or open [the interactive browser edition](reader.html) to change the examples. No code editing is needed.\n",
"\n",
"Request an explicit feature representation, its units and distance, a target population or target coverage list, and generated examples. When probabilities are supplied, compare mode mass and transport or JS using that defined support. Without a target distribution or explicit features, return a qualitative coverage checklist and missing-information request. For Gaussian latent models request prior, observation variance, observation and variational mean/width; report reconstruction, prior KL, evidence and posterior gap. For flows verify invertibility and a nonzero Jacobian before transforming density."
]
},
{
"cell_type": "markdown",
"id": "3873e403",
"metadata": {},
"source": [
"Reproducing these calculations
\n",
"\n",
"The optional calculation cells use Python with NumPy, SciPy, and Matplotlib. All inputs are included in this file. The figures below are saved, so running these cells is optional.\n",
"\n",
"Show the answer
\n",
"\n",
"The total is 9 + 0.0625 = 9.0625 and the hidden part is 9, so the share is 9 / 9.0625, about 99.3%.\n",
"\n",
"Show the answer
\n",
"\n",
"q = N(0.8, 0.2), SD 0.447. The gap becomes 0 and the ELBO equals the log evidence, but the prior KL stays positive (0.725).\n",
"\n",
"Show the answer
\n",
"\n",
"Half the mass must move a distance 2, so W1 = 0.5 x 2 = 1 feature unit. The right mode has zero coverage.\n",
"\n",
"Show the answer
\n",
"\n",
"The endpoints are 3 and 1, so the image is [1, 3]. Its mass is Phi(1) - Phi(0), about 0.341, despite the reversal.\n",
"\n",
"Show the answer
\n",
"\n",
"JS approaches ln 2 = 0.693, so V approaches -1.386 + 2 x 0.693 = 0. The game value is near its ceiling and flat in the generator position.\n",
"\n",
"Show the answer
\n",
"\n",
"Every weight then equals p(x) exactly, so the average of M weights is the same constant and L_M equals the log evidence for every M.\n",
"\n",
"Show the answer
\n",
"\n",
"Every path goes to +2, so the velocity is (2 - 0.5) / (1 - 0.5) = 3 for all of them, and the learned velocity is 3.\n",
"\n",
"Show the answer
\n",
"\n",
"Only the spread term: with scale 0.5 it is (4 + 0.49) x (1 - 0.5)^2 = 1.12, while the centre gap is 0 if the centres agree.\n",
"\n",
"