{
"cells": [
{
"cell_type": "markdown",
"id": "f9ad9a95",
"metadata": {},
"source": [
"# Robustness, OOD, and Causality\n",
"\n",
"*The Mathematics of Large Language Models ยท Chapter 15*\n",
"\n",
"Check how a conclusion changes with inputs, populations and interventions.\n",
"\n",
"These pages illustrate the mathematics. Read the saved figures and calculations in order, or open [the interactive browser edition](reader.html) to change the examples. No code editing is needed.\n",
"\n",
"Identify the kind of change first. Numerical perturbations, population shifts and interventions need different assumptions and support different conclusions."
]
},
{
"cell_type": "markdown",
"id": "f2185d3c",
"metadata": {},
"source": [
"Reproducing these calculations
\n",
"\n",
"The optional calculation cells use Python with NumPy, SciPy, and Matplotlib. All inputs are included in this file. The figures below are saved, so running these cells is optional.\n",
"\n",
"Show the answer
\n",
"\n",
"The score drops by 3 x 0.1 = 0.3 to 0.2, still positive, so the label stays. It would flip at a budget above 0.5 / 3, about 0.167.\n",
"\n",
"Show the answer
\n",
"\n",
"The bound is 0.01^(1/1000), about 0.9954, and 0.2 x Phi^-1(0.9954) = 0.2 x 2.61, about 0.52.\n",
"\n",
"Show the answer
\n",
"\n",
"Overall 0.5 x 0.1 + 0.5 x 0.4 = 0.25; the worst group is B at 0.4. Only when B has zero share is the worst group present A at 0.1.\n",
"\n",
"Show the answer
\n",
"\n",
"Seen: 0.5 - 0.2 = 0.3; adjusted: both are 0.35, so the effect is 0 under the stipulated graph.\n",
"\n",
"Show the answer
\n",
"\n",
"No. The formula needs U to have no path into M, otherwise averaging over X does not remove the confounding of M and Y.\n",
"\n",
"Show the answer
\n",
"\n",
"The causal weight 1 / (1 + tau^2) and spurious weight 0: at an even mix the copy has zero correlation with the outcome itself, so it gets no weight.\n",
"\n",
"Show the answer
\n",
"\n",
"The model improves on the heavily weighted groups, so their loss falls, which pulls the weights back toward balance.\n",
"\n",
"Show the answer
\n",
"\n",
"Half-width (0.7 + 0.7) x 0.1 = 0.14, and this is exact for a single layer because both inputs can move together.\n",
"\n",
"