{
"cells": [
{
"cell_type": "markdown",
"id": "d71bf82a",
"metadata": {},
"source": [
"# Symmetry and Geometric Inductive Bias\n",
"\n",
"*The Mathematics of Large Language Models ยท Chapter 5*\n",
"\n",
"Specify what should stay fixed and what should transform.\n",
"\n",
"These pages illustrate the mathematics. Read the saved figures and calculations in order, or open [the interactive browser edition](reader.html) to change the examples. No code editing is needed.\n",
"\n",
"Identify the input space, transformation group, output type and desired transformation law. Return explicit matched inputs, expected scalar invariance or vector equivariance, and numerical discrepancy measures when outputs are available. Treat semantic invariance as a hypothesis to audit."
]
},
{
"cell_type": "markdown",
"id": "1f8a1716",
"metadata": {},
"source": [
"Reproducing these calculations
\n",
"\n",
"The optional calculation cells use Python with NumPy, SciPy, and Matplotlib. All inputs are included in this file. The figures below are saved, so running these cells is optional.\n",
"\n",
"Show the answer
\n",
"\n",
"The new arrow is (0, -3) and the length is still 3: a half turn flips the direction and keeps the size.\n",
"\n",
"Show the answer
\n",
"\n",
"No, only every 180 degrees. Here f(-x) = f(x), so the two-turn average is just f, which is 1 at angle 0 and 2 at angle 90 (radius 1).\n",
"\n",
"Show the answer
\n",
"\n",
"Values slid past the crop are thrown away and zero padding brings in different border values. Compare interiors only, or use an explicitly cyclic domain.\n",
"\n",
"Show the answer
\n",
"\n",
"Sum goes from 7 to 11, mean from 7/3 to 11/4, and the maximum stays 4. All three ignore order, but they keep different information about repeats.\n",
"\n",
"Show the answer
\n",
"\n",
"Multiply by e^(-2 pi i x 3/4) = i, giving -0.5 + 0.5i. The size is 0.707 both before and after.\n",
"\n",
"Show the answer
\n",
"\n",
"Three places in the same direction (equivalently one place back), because 270 degrees is three quarter turns.\n",
"\n",
"Show the answer
\n",
"\n",
"Equivariant: 10 x 10 + 10 x 10 = 200. Dense: the input has 10 + 3 x 10 = 40 numbers, so 40 x 40 = 1,600, which is 8 times more.\n",
"\n",
"Show the answer
\n",
"\n",
"The force turns with the molecule, so it becomes (0, 0.4, 0). Its size, 0.4, is unchanged.\n",
"\n",
"