{
"cells": [
{
"cell_type": "markdown",
"id": "310ab122",
"metadata": {},
"source": [
"# Bayesian Deep Learning and Uncertainty\n",
"\n",
"*The Mathematics of Large Language Models ยท Chapter 14*\n",
"\n",
"Update a belief, separate uncertainties, and check a forecast.\n",
"\n",
"These pages illustrate the mathematics. Read the saved figures and calculations in order, or open [the interactive browser edition](reader.html) to change the examples. No code editing is needed.\n",
"\n",
"Use resolved records to assess forecasts, or a stated probabilistic model to update a belief. Each method needs different inputs; a single confident sentence does not supply a calibrated probability."
]
},
{
"cell_type": "markdown",
"id": "e7bf95bb",
"metadata": {},
"source": [
"Reproducing these calculations
\n",
"\n",
"The optional calculation cells use Python with NumPy, SciPy, and Matplotlib. All inputs are included in this file. The figures below are saved, so running these cells is optional.\n",
"\n",
"Show the answer
\n",
"\n",
"Posterior Beta(10, 4); mean 10/14 = 5/7, about 0.714.\n",
"\n",
"Show the answer
\n",
"\n",
"Between variance (1 + 0 + 1)/3 = 2/3; total 5/3, not 2.\n",
"\n",
"Show the answer
\n",
"\n",
"|0.6 - 0.8| = 0.2; copying all rows leaves the same gap.\n",
"\n",
"Show the answer
\n",
"\n",
"ceil(5 x 0.9) = 5 > 4, so q is infinity; an interpolated finite quantile would be wrong.\n",
"\n",
"Show the answer
\n",
"\n",
"The nearest dot is 2.1 away, so every kernel value is about exp(-24.5), nearly 0. The mean returns to the prior mean 0 and the sd to the prior sd 1.\n",
"\n",
"Show the answer
\n",
"\n",
"Exploring: sqrt(beta) multiplies the sd, going from 2 to 3, so uncertain places get a bigger bonus.\n",
"\n",
"Show the answer
\n",
"\n",
"Above 1, to soften the probabilities. Accuracy stays 70 percent because dividing scores by a positive number never changes their order.\n",
"\n",
"Show the answer
\n",
"\n",
"Both cover time 20. Step 0.5 has the smaller swing: its energy stays within 1/(1 - 0.0625) = 1.07 times the start against 1.33 for step 1.\n",
"\n",
"