Mathematical Rules of Thumb, illustrated reader · Chapter 20

20Numerical Methods

Approximation You Can Trust

5 demonstrations follow the chapter's rules. Choose a value, watch the figure and the numbers change, and check your prediction. Every choice is precomputed from the notebook calculations.

Ask the chapter skill

“Help me use Chapter 20 for my question. Choose a rule, check its assumptions, and show how the result changes when an input changes.”

Use math-thumb-numerical-methods from the companion's skill package. The demonstrations below also work on their own.

Examples use constructed inputs or the book's own values, disclosed in each panel. A picture illustrates a rule; its assumptions set its scope.

1Demonstration 1 of 5

Turn a bracket into a numerical certificate

What does one extra halving buy?

Actually bisect x²−2 on [1,2], then compare the observed error with the guaranteed midpoint ceiling.

|xmid−x*|≤b0−a02k+1 |x_{mid}-x_*|\leq\frac{b_0-a_0}{2^{k+1}}

Bisection halvings k. Continuous function and a sign-changing bracket containing the positive root.

Predict first. What does one extra halving buy?

Choose an example

Turn a bracket into a numerical certificate. 10 halvings reduce a width-one bracket to 0.000976562. Continuity and a valid root bracket give the midpoint error ceiling 0.000488281.
Bisection halvings k: 10
Constructed teaching inputs; calculations executed locally. Supported menu choices are precomputed.

Calculated values

Halvings
10
Midpoint
1.41455
Actual error
0.000337219
Certified error ceiling
0.000488281

10 halvings reduce a width-one bracket to 0.000976562. Continuity and a valid root bracket give the midpoint error ceiling 0.000488281.

Use the idea

Use rule 20.3.5 when its stated conditions fit. Compare the calculation with your own decision threshold; retain the relevant error or uncertainty.

Where the conclusion applies

Continuous function and a sign-changing bracket containing the positive root.

Check your understanding: What does one extra halving buy?
It halves the bracket width and the midpoint error ceiling.

Book source: Rule 20.3.5: Budget bisection iterations from the bracket width. Demonstration C20-D01. Worked illustration.

2Demonstration 2 of 5

Find where a smaller derivative step stops helping

Why does extremely tiny h fail?

Forward and centered differences encounter both truncation and floating subtraction errors.

f′(x)≈f(x+h)−f(x−h)2h f^{\prime}(x)\approx\frac{f(x+h)-f(x-h)}{2h}

Difference step h. Differentiate exp(x) at x=1 in binary64. Function scale and derivatives influence the best step.

Predict first. Why does extremely tiny h fail?

Choose an example

Find where a smaller derivative step stops helping. At h=1e-08, forward difference: it is near its best step; centered difference: rounding dominates, so a smaller step makes it worse. Optimal step scales depend on precision, derivative size and function implementation, not a universally best constant.
Difference step h: 1e-08
Constructed teaching inputs; calculations executed locally. Supported menu choices are precomputed.

Calculated values

Step h
1e-08
Forward error
6.60275e-09
Centered error
6.60275e-09

At h=1e-08, forward difference: it is near its best step; centered difference: rounding dominates, so a smaller step makes it worse. Optimal step scales depend on precision, derivative size and function implementation, not a universally best constant.

Use the idea

Use rule 20.2.2 when its stated conditions fit. Compare the calculation with your own decision threshold; retain the relevant error or uncertainty.

Where the conclusion applies

Differentiate exp(x) at x=1 in binary64. Function scale and derivatives influence the best step.

Check your understanding: Why does extremely tiny h fail?
The nearby function values round before subtraction, so dividing by tiny h amplifies rounding error.

Book source: Rule 20.2.2: Use a cube-root-epsilon step for centered differences. Demonstration C20-D02. Worked illustration.

3Demonstration 3 of 5

Measure trapezoid convergence

What error ratio should step halving approach?

Compute the integral on successive meshes and compare with the known exact value.

∫01exdx=e−1,E(h)=O(h2) \int_0^1e^x dx=e-1,\quad E(h)=O(h^2)

Subinterval count n. Smooth integrand, uniform composite trapezoid rule.

Predict first. What error ratio should step halving approach?

Choose an example

Measure trapezoid convergence. The smooth integral of exp(x) on [0,1] is e−1. Doubling the subinterval count decreases trapezoid error by about four; nonsmooth integrands need separate analysis.
Subinterval count n: 32
Constructed teaching inputs; calculations executed locally. Supported menu choices are precomputed.

Calculated values

Subintervals
32
Absolute error
0.000139832
Error ratio when n doubles (previous / selected)
3.9998

The smooth integral of exp(x) on [0,1] is e−1. Doubling the subinterval count decreases trapezoid error by about four; nonsmooth integrands need separate analysis.

Use the idea

Use rule 20.3.1 when its stated conditions fit. Compare the calculation with your own decision threshold; retain the relevant error or uncertainty.

Where the conclusion applies

Smooth integrand, uniform composite trapezoid rule.

Check your understanding: What error ratio should step halving approach?
About four in the second-order regime, before roundoff dominates.

Book source: Rule 20.3.1: Expect a factor of four from halving trapezoid spacing. Demonstration C20-D03. Worked illustration.

4Demonstration 4 of 5

Watch equal spacing blow up

Does raising the degree always help?

Interpolate the same smooth bump with equally spaced points and with Chebyshev points.

f(x)=11+25x2,xj=cos⁡(2j+1)π2n+2 f(x)=\frac{1}{1+25x^2},\quad x_j=\cos\frac{(2j+1)\pi}{2n+2}

Polynomial degree n. Exact data, one global polynomial on [-1,1] through n+1 nodes, binary64.

Predict first. Does raising the degree always help?

Choose an example

Watch equal spacing blow up. Degree 12: worst error 3.66 with equal spacing, 0.0692 with Chebyshev nodes. Raising the degree makes equal spacing worse while Chebyshev keeps improving.
Polynomial degree n: 12
Constructed teaching inputs; calculations executed locally. Supported menu choices are precomputed.

Calculated values

Degree
12
Max error, equispaced
3.66326
Max error, Chebyshev
0.0692157

Degree 12: worst error 3.66 with equal spacing, 0.0692 with Chebyshev nodes. Raising the degree makes equal spacing worse while Chebyshev keeps improving.

Use the idea

Use rule 20.2.4 when its stated conditions fit. Compare the calculation with your own decision threshold; retain the relevant error or uncertainty.

Where the conclusion applies

Exact data, one global polynomial on [-1,1] through n+1 nodes, binary64.

Check your understanding: Does raising the degree always help?
Not with equal spacing: worst error grows from .62 to 29. Chebyshev nodes shrink it from .26 to .022.

Book source: Rule 20.2.4: Use Chebyshev-like nodes for high-degree global interpolation. Demonstration C20-D04. Worked illustration.

5Demonstration 5 of 5

Recover small terms with compensated summation

Does adding a million copies of 1e-16 to 1 in plain floating point change the total?

Add many tiny terms to 1. A plain running sum rounds each one away; a compensated sum keeps a correction term and recovers them.

Kahan: y=ak−c,t=s+y,c=(t−s)−y,s=t \text{Kahan: } y=a_k-c,\ t=s+y,\ c=(t-s)-y,\ s=t

Number of 1e-16 terms. Binary64, round to nearest, terms below half an ulp of the running total.

Predict first. Does adding a million copies of 1e-16 to 1 in plain floating point change the total?

Choose an example

Recover small terms with compensated summation. Start at 1 and add 100,000 copies of 1e-16. The exact total grows by 1e-11, but each 1e-16 is below half the spacing of doubles near 1, so the plain sum rounds every one away and stays exactly 1. The compensated sum carries the lost bits forward and gets 1e-11. The plain total is off by 1e-11 in relative terms, and the gap grows with every term; Kahan stays at rounding level.
Number of 1e-16 terms: 100000
Constructed teaching inputs; calculations executed locally. Supported menu choices are precomputed.

Calculated values

Tiny terms added
100,000
Exact sum minus 1
1e-11
Plain sum minus 1
0
Kahan sum minus 1
1e-11

Start at 1 and add 100,000 copies of 1e-16. The exact total grows by 1e-11, but each 1e-16 is below half the spacing of doubles near 1, so the plain sum rounds every one away and stays exactly 1. The compensated sum carries the lost bits forward and gets 1e-11. The plain total is off by 1e-11 in relative terms, and the gap grows with every term; Kahan stays at rounding level.

Use the idea

Use rule 20.1.3 when its stated conditions fit. Compare the calculation with your own decision threshold; retain the relevant error or uncertainty.

Where the conclusion applies

Binary64, round to nearest, terms below half an ulp of the running total.

Check your understanding: Does adding a million copies of 1e-16 to 1 in plain floating point change the total?
No. Each addition rounds back to exactly 1, losing the whole 1e-10. Kahan summation recovers it.

Book source: Rule 20.1.3: Use compensated summation for long mixed-scale sums. Demonstration C20-D05. Worked illustration.

Bring the idea to a question of your own

Choose the relationship that answers your question, check its conditions, and compare the result with the accuracy or decision threshold you need.

The chapter skill can adapt these calculations to your inputs. It should name the assumptions, explain what the result supports, and say what still needs evidence. The chapter workbook adds a lab and three exercises with answers.