--- name: math-thumb-optimization description: "Apply Chapter 19 (Optimization: Scale, Search, and Certify) of Mathematical Rules of Thumb to solve, check, or teach problems. Use it to choose a step and stopping test that match the optimization problem's assumptions." --- # Optimization: Scale, Search, and Certify Use this chapter to help the reader make a checked mathematical decision. All 19 numbered rules are available in [the chapter source](references/chapter.md). [The workbook](references/notebook.md) contains a lab, exercises, solutions, and the full rule checklist. [The local rule index](references/rules.json) supplies discovery metadata. ## Start from the reader's task Infer solve, learn, or audit mode from the request. In solve mode, use their supplied numbers and target; in learn mode, use the workbook or their chosen rule; in audit mode, inspect their actual calculation before replacing it. Gather only missing information that changes the choice: Objective and constraints; units and variable scales; smoothness and convexity evidence; derivative access; noise; required optimality certificate. If the question falls outside this chapter, say which mathematical operation is missing and suggest a relevant chapter. If the whole-book skill is available, it can carry the task onward, but this chapter works independently. ## Select and apply a rule - **Does the problem mix units or characteristic magnitudes?** Nondimensionalize variables and constraints, then transform derivatives, bounds, penalties, and reported sensitivities consistently. - **Can the objective be evaluated safely?** Stabilize exponentials, logarithms, and reductions before diagnosing optimizer behavior. - **Is the objective smooth and convex?** Use a reciprocal smoothness step or backtracking; use condition number to judge whether scaling, acceleration, or curvature is needed. - **Are gradients noisy?** Identify the batch-variance and constant-step floor before tightening tolerances or declaring a bug. - **Is second-order information available?** Globalize Newton with a line search or trust region; use L-BFGS when only a small vector history fits. - **Is the landscape nonconvex?** Treat every local solve as one basin sample and escalate to global methods when a certificate matters. - **Have derivatives been independently checked?** Sweep centered directional differences across several steps before trusting convergence behavior. - **What does optimality mean here?** Use a duality gap for valid convex bounds, a projected mapping for closed convex sets, or full scaled KKT blocks for general constraints. - **Is the diagnostic algorithm-specific?** Keep Newton decrement, ADMM balancing, and penalty continuation attached to their required method and assumptions. - **Will a multiplier drive a decision?** Verify the active set, units, signs, regularity, and locality before interpreting it as marginal value. Read the selected complete profile, including its equation, “How to read it,” and “How to use it.” The compact graph assumptions are search cues, not a substitute for the profile. Preserve the numbered citation and role. **Independent**, **Workflow**, and **Specialized** describe the relationship to a calculation; exactness, approximation, bound, diagnostic, and heuristic describe a different dimension. Use verified inputs, show the substitution and units, and interpret the result in the reader's decision. Verify by an appropriate bound, alternative computation, limiting case, residual with conditioning, or sensitivity check. If a required condition fails, reject that use and give the specific missing information or alternative method; do not calculate a plausible-looking answer from an invalid formula. **Essential boundary:** Keep convergence claims attached to convexity, smoothness, scaling, and method conditions. Use constraint-aware optimality residuals at boundaries. For a sufficient independent result, stop with the decision it supports. For a workflow or specialized rule, name the downstream calculation still needed. A numerical demonstration is evidence for that instance, not a universal proof. ## Teach and check understanding Use the [workbook](references/notebook.md) for guided practice. Start with 19.2.1, 19.3.1, 19.3.3 when the reader wants a starting exercise. Ask for an attempt, offer a relevant hint, and reveal the answer when requested or when teaching requires it. Do not force a quiz when the reader asked for a worked solution. Check whether the reader can explain the controlling quantity, apply the rule to a changed input, identify an invalid use, and distinguish a final answer from a preparatory step. Track only demonstrated work. Give a short prerequisite explanation when needed; avoid requiring completion of earlier chapters. ## Return a usable result Include the chosen rule numbers, assumptions that matter, calculation, verification, and next action. For ongoing work, offer this compact record: question; inputs and units; rules; claim type; book role; assumption status; result and error; check; decision; unresolved next step. Write a progress file only when asked or within an already authorized notebook-editing task. The source chapter is a fixed book snapshot. Preserve its mathematical qualifications and historical evidence gaps. Use outside material only when the reader's task needs it, verify material facts appropriately, and identify that material separately from the book. ## Illustrated exploration Open [the browser reader](assets/reader.html) or [the saved illustrated notebook](assets/notebook.ipynb). [Equation cards](references/equations.json) record the book rule, formula, fixed inputs, supported choices, assumptions and executed default results. - **C19-D01: Run gradient descent on a stiff quadratic**: rule 19.2.1. - **C19-D02: Shift exponentials without changing softmax**: rule 19.1.3. - **C19-D03: Use a feasible stopping test at a boundary**: rule 19.3.3. - **C19-D04: See why condition number sets the pace**: rule 19.1.4. - **C19-D05: See constant-step noise set a floor**: rule 19.1.5. Use a saved illustration only when its conditions fit. Browser controls select finite precomputed choices; they do not calculate arbitrary reader inputs. For different inputs, make a checked calculation using the selected rule. Explain what changes, and never claim the notebook ran or the browser was viewed unless it did. Offer prediction questions for learning; answer direct requests without a mandatory quiz.