{
"cells": [
{
"cell_type": "markdown",
"id": "9ab0eb95",
"metadata": {},
"source": [
"# Convolutional and Multiscale Models\n",
"\n",
"*The Mathematics of Large Language Models ยท Chapter 6*\n",
"\n",
"Inspect filter conventions, aliases, localized changes, residual maps, and the cost of scale.\n",
"\n",
"These pages illustrate the mathematics. Read the saved figures and calculations in order, or open [the interactive browser edition](reader.html) to change the examples. No code editing is needed.\n",
"\n",
"Request sampling interval and units, raw data, boundary convention, frequency range, and whether filtering preceded sampling. Return raw and filtered views, appropriate frequency or wavelet calculations, and stated edge, lag and aliasing limitations. Use residual dynamics only for a specified numeric map."
]
},
{
"cell_type": "markdown",
"id": "6dc745e3",
"metadata": {},
"source": [
"Reproducing these calculations
\n",
"\n",
"The optional calculation cells use Python with NumPy, SciPy, and Matplotlib. All inputs are included in this file. The figures below are saved, so running these cells is optional.\n",
"\n",
"Show the answer
\n",
"\n",
"The first gives 6 x (1/2 + 1/2) = 6, a pure smoother. The second gives 6 x (1 - 2 + 1) = 0, because its weights sum to zero. The zero-frequency gain is always the sum of the weights.\n",
"\n",
"Show the answer
\n",
"\n",
"30 Hz: 130 - 1 x 100 = 30, which is below 50 Hz. The hum cannot be told from a real 30 Hz signal afterwards, so the signal must be filtered before it is sampled.\n",
"\n",
"Show the answer
\n",
"\n",
"Substituting z = (x - b) / a turns the overlap into the constant times sqrt(a) times the area of the wavelet, which is zero. Cutting the window short can spoil the cancellation.\n",
"\n",
"Show the answer
\n",
"\n",
"One block multiplies it by 1 - 2/4 = 0.5; four blocks give 0.5^4 = 0.0625. The smooth flow gives exp(-2) = 0.135, so four blocks are too coarse.\n",
"\n",
"Show the answer
\n",
"\n",
"Frequency spread = 1 / (4 pi x 0.5) = 0.159, since the Gaussian attains the limit. No other shape can have a smaller product, so any other shape with position spread 0.5 has a larger frequency spread.\n",
"\n",
"Show the answer
\n",
"\n",
"None: every pair has difference zero. The single average number rebuilds it, since each step just copies the average to both halves.\n",
"\n",
"Show the answer
\n",
"\n",
"Up to about 32 samples. Raising J by one doubles the window, so the bound c / 2^J halves for the same shift, at the price of less position detail.\n",
"\n",
"Show the answer
\n",
"\n",
"N = a / b = 0.0860 / 0.0000195, about 4400 patches, so the side is 16 x sqrt(4400), about 1060 pixels. Beyond that, attention is more than half of the cost.\n",
"\n",
"