{
"cells": [
{
"cell_type": "markdown",
"id": "c59a5d4ade12",
"metadata": {},
"source": [
"# Chapter 10: Plans Within Plans: Hierarchical Actions and Reusable Skills\n",
"\n",
"A reusable review skill hides several primitive actions behind one name. That abstraction is useful, but it can also hide time. A one-step archive action and a three-step review-release option do not place their continuation at the same discount offset. Nor does beginning an option imply that its termination condition will be reached.\n",
"\n",
"This notebook compares declared options through their reward traces, initiation predicates, completion flags, and deadlines. It does not execute a live skill. It calculates what follows from a finite trace. The changed case interrupts the useful option before its release reward, forcing the report to separate local activity from workflow completion.\n",
"\n",
"**Outcome:** Compute option returns with duration discounting and explicit completion.\n",
"\n",
"- Inspect the declared input contract\n",
"- Predict the hand-checkable case\n",
"- Run the shared computation\n",
"- Change the critical assumption\n",
"- Apply the method to the transfer data\n",
"\n",
"**Guided route:** Run the worked calculation, inspect its figure, change the stated assumption, and try the transfer case. Read the explanations beside each result before opening the answers.\n",
"\n",
"**Deeper route:** First read the mathematics and canonical equation reference. Audit the input contract, predict the changed result, then inspect the shared chapter implementation and solve the questions independently. Both routes use the same calculations and preserve the equations."
]
},
{
"cell_type": "markdown",
"id": "a7781f3ee7dc",
"metadata": {},
"source": [
"## Technical Requirements\n",
"\n",
"Python 3.11 or later, the complete laboratory folder, and the notebook dependencies listed in `requirements-notebooks.txt` (the launcher's **Install notebook tools** choice installs them; see START-HERE). Standard-library chapter commands also support Python 3.10. No API key, model account or network call is used by this experiment.\n",
"\n",
"Prior knowledge:\n",
"\n",
"- Python lists and dictionaries\n",
"- The mapped chapter and its notation"
]
},
{
"cell_type": "markdown",
"id": "5651ec848c83",
"metadata": {},
"source": [
"## The question and its mathematics\n",
"\n",
"For an option of duration d with primitive rewards r_0,...,r_(d-1), its completed value is sum_(t=0)^(d-1) gamma^t r_t + gamma^d V_cont. The continuation discount depends on duration, not on the fact that the option has one convenient label.\n",
"\n",
"The option is executable only when its initiation predicate holds. Its executed prefix is limited by remaining deadline and the declared interruption cap. Continuation value is added only if every primitive step executes and the termination flag is true. An executed prefix may have nonzero reward while completion remains false.\n",
"\n",
"This is a deterministic trace calculation. It does not model a distribution of option durations or an internal stochastic policy. Those would require expected discounted rewards and state-dependent termination probabilities. The finite representation is still useful for catching a common modeling error: assigning the full continuation value to a macro-action that never reaches its promised boundary."
]
},
{
"cell_type": "markdown",
"id": "421a4ffdd139",
"metadata": {},
"source": [
"## A calculation you can run\n",
"\n",
"Each option row supplies a name, exact boolean initiation status, primitive rewards, exact boolean termination status, and optional continuation value. Global inputs specify discount, deadline, elapsed start time, and optional interruption cap. Validation rejects invalid counts, nonfinite rewards, and ambiguous truth values.\n",
"\n",
"The function determines the available prefix, sums its discounted rewards, and conditionally adds completion continuation. It reports full duration, executed duration, completion, and continuation discount. The figures compare executed value and primitive-step count; they should not be combined because their units differ. Run the changed interruption case and inspect which reward terms disappeared. A high full-option value is irrelevant if the actual prefix cannot reach it.\n",
"\n",
"The next cell finds the bundle and imports the same computation used by the chapter skill. It does not change your system Python."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "42ebefc0e5c1",
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"import sys, json\n",
"LAB_ROOT = next((p for p in [Path.cwd(), *Path.cwd().parents] if (p / \"lab-manifest.json\").is_file()), None)\n",
"if LAB_ROOT is None:\n",
" raise RuntimeError(\"Open this notebook from the complete extracted laboratory folder.\")\n",
"sys.path.insert(0, str(LAB_ROOT / \"src\"))\n",
"from math_ai_agents.core import analyze, report_text\n",
"from math_ai_agents.plotting import figure_svg\n",
"from IPython.display import SVG, display\n"
]
},
{
"cell_type": "markdown",
"id": "7fffde8c0edb",
"metadata": {},
"source": [
"Set the declared inputs below. These are constructed teaching values, not measurements from a production agent. Change a value only after predicting what it should change."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "b74077119e3c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chapter 10: option-duration\n",
"Does a reusable skill finish before interruption or the deadline?\n",
"Evidence: constructed teaching example\n",
"\n",
"Calculated quantities:\n",
"{\n",
" \"options\": [\n",
" {\n",
" \"name\": \"review-release\",\n",
" \"duration\": 3,\n",
" \"initiated\": true,\n",
" \"executed\": 3,\n",
" \"complete\": true,\n",
" \"value\": 6.038,\n",
" \"continuation_discount\": 0.729\n",
" },\n",
" {\n",
" \"name\": \"archive\",\n",
" \"duration\": 1,\n",
" \"initiated\": true,\n",
" \"executed\": 1,\n",
" \"complete\": true,\n",
" \"value\": 1.0,\n",
" \"continuation_discount\": 0.9\n",
" }\n",
" ],\n",
" \"best_executed_value\": \"review-release\",\n",
" \"completed_options\": 2\n",
"}\n",
"\n",
"Interpretation:\n",
"An option receives continuation value only after its declared termination. A deadline or interruption can leave locally useful work without workflow completion.\n",
"\n",
"Assumptions:\n",
"- Rewards occur at primitive time offsets beginning at zero.\n",
"- Continuation discount uses the full option duration.\n",
"- Initiation and termination predicates are declared for this state.\n",
"\n",
"Limitations:\n",
"- The finite reward trace is not a learned option policy.\n",
"- An interrupted option needs a recovery contract before reuse.\n",
"\n",
"Execution: completed locally; constructed inputs are not deployment measurements.\n"
]
}
],
"source": [
"chapter = 10\n",
"inputs = {'discount': 0.9,\n",
" 'deadline': 5,\n",
" 'options': [{'name': 'review-release',\n",
" 'initiation': True,\n",
" 'rewards': [-1, -1, 8],\n",
" 'terminated': True,\n",
" 'continuation_value': 2},\n",
" {'name': 'archive',\n",
" 'initiation': True,\n",
" 'rewards': [1],\n",
" 'terminated': True,\n",
" 'continuation_value': 0}]}\n",
"report = analyze(chapter, inputs)\n",
"# This input was explicitly taken from the teaching fixture.\n",
"report['evidence_kind'] = 'constructed teaching example'\n",
"print(report_text(report))"
]
},
{
"cell_type": "markdown",
"id": "7dc5fc8ac4d0",
"metadata": {},
"source": [
"The default review-release value is -1-0.9+0.9^2(8)+0.9^3(2)=6.038. It completes in three steps within the five-step deadline. Archive gives value 1.\n",
"\n",
"The changed interruption permits only two steps of review-release. Its value becomes -1-0.9=-1.9, completion is false, and no continuation is added. Archive still completes and remains worth 1, so it becomes the preferred executed value. The reversal comes from lost access to the option's terminal benefit, not a change in the release payoff.\n",
"\n",
"The plot below uses the calculated quantities. Read each panel's units before comparing its values."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "590f57e74048",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Matplotlib is building the font cache; this may take a moment.\n"
]
},
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display(SVG(figure_svg(report)))"
]
},
{
"cell_type": "markdown",
"id": "7208c3d00715",
"metadata": {},
"source": [
"**Figure 10.L1:** Calculated chapter experiment. Each panel labels its input and output units; interpret it under the assumptions printed in the report."
]
},
{
"cell_type": "markdown",
"id": "9c1a1bffcf08",
"metadata": {},
"source": [
"## Change the assumption\n",
"\n",
"An option label can conceal an unsafe initiation assumption. If review requires a current document version, a generic initiation flag is insufficient for deployment until that predicate is tied to observed state. Likewise, the returned termination flag is a supplied trace fact, not an automatically verified effect.\n",
"\n",
"Another failure is to discount continuation by one macro step regardless of duration. That exaggerates long options when gamma is below one. The transfer case exposes the difference directly.\n",
"\n",
"An interruption contract should state what remains durable, whether the option can resume, and which action is safe next. This notebook stops at the prefix and does not invent recovery. A partial option can create cost or state change even when its workflow completion flag is false. Preserve that consequence in a larger trace model."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "be203c6e56cf",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chapter 10: option-duration\n",
"Does a reusable skill finish before interruption or the deadline?\n",
"Evidence: constructed changed-assumption example\n",
"\n",
"Calculated quantities:\n",
"{\n",
" \"options\": [\n",
" {\n",
" \"name\": \"review-release\",\n",
" \"duration\": 3,\n",
" \"initiated\": true,\n",
" \"executed\": 2,\n",
" \"complete\": false,\n",
" \"value\": -1.9,\n",
" \"continuation_discount\": 0\n",
" },\n",
" {\n",
" \"name\": \"archive\",\n",
" \"duration\": 1,\n",
" \"initiated\": true,\n",
" \"executed\": 1,\n",
" \"complete\": true,\n",
" \"value\": 1.0,\n",
" \"continuation_discount\": 0.9\n",
" }\n",
" ],\n",
" \"best_executed_value\": \"archive\",\n",
" \"completed_options\": 1\n",
"}\n",
"\n",
"Interpretation:\n",
"An option receives continuation value only after its declared termination. A deadline or interruption can leave locally useful work without workflow completion.\n",
"\n",
"Assumptions:\n",
"- Rewards occur at primitive time offsets beginning at zero.\n",
"- Continuation discount uses the full option duration.\n",
"- Initiation and termination predicates are declared for this state.\n",
"\n",
"Limitations:\n",
"- The finite reward trace is not a learned option policy.\n",
"- An interrupted option needs a recovery contract before reuse.\n",
"\n",
"Execution: completed locally; constructed inputs are not deployment measurements.\n"
]
},
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"changed_inputs = {'discount': 0.9,\n",
" 'deadline': 5,\n",
" 'options': [{'name': 'review-release',\n",
" 'initiation': True,\n",
" 'rewards': [-1, -1, 8],\n",
" 'terminated': True,\n",
" 'continuation_value': 2},\n",
" {'name': 'archive',\n",
" 'initiation': True,\n",
" 'rewards': [1],\n",
" 'terminated': True,\n",
" 'continuation_value': 0}],\n",
" 'interrupt_after': 2}\n",
"changed = analyze(chapter, changed_inputs)\n",
"changed['evidence_kind'] = 'constructed changed-assumption example'\n",
"print(report_text(changed))\n",
"display(SVG(figure_svg(changed)))"
]
},
{
"cell_type": "markdown",
"id": "b8721487d6f8",
"metadata": {},
"source": [
"**Figure 10.L2:** The changed-assumption result. Compare the printed quantities and the stated assumptions with the first run. A different input need not imply a causal effect in a deployed agent."
]
},
{
"cell_type": "markdown",
"id": "4ed3e390cdfe",
"metadata": {},
"source": [
"## Try a new case\n",
"\n",
"In the transfer case, inspect executes two steps with rewards 0 and 4. At gamma=0.5, its local reward is 2 and its continuation contributes 0.5^2(8)=2, for total value 4. The disabled option executes zero steps despite its attractive nominal reward.\n",
"\n",
"For local use, extract primitive duration and reward timing from a compatible declared workflow. Keep initiation, termination, interruption, and deadline conditions explicit. If durations vary, analyze separate traces or extend the model with a documented distribution. Do not replace measured completion with a skill's descriptive name."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "f5607100a8b8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chapter 10: option-duration\n",
"Does a reusable skill finish before interruption or the deadline?\n",
"Evidence: constructed transfer example\n",
"\n",
"Calculated quantities:\n",
"{\n",
" \"options\": [\n",
" {\n",
" \"name\": \"inspect\",\n",
" \"duration\": 2,\n",
" \"initiated\": true,\n",
" \"executed\": 2,\n",
" \"complete\": true,\n",
" \"value\": 4.0,\n",
" \"continuation_discount\": 0.25\n",
" },\n",
" {\n",
" \"name\": \"disabled\",\n",
" \"duration\": 1,\n",
" \"initiated\": false,\n",
" \"executed\": 0,\n",
" \"complete\": false,\n",
" \"value\": 0,\n",
" \"continuation_discount\": 0\n",
" }\n",
" ],\n",
" \"best_executed_value\": \"inspect\",\n",
" \"completed_options\": 1\n",
"}\n",
"\n",
"Interpretation:\n",
"An option receives continuation value only after its declared termination. A deadline or interruption can leave locally useful work without workflow completion.\n",
"\n",
"Assumptions:\n",
"- Rewards occur at primitive time offsets beginning at zero.\n",
"- Continuation discount uses the full option duration.\n",
"- Initiation and termination predicates are declared for this state.\n",
"\n",
"Limitations:\n",
"- The finite reward trace is not a learned option policy.\n",
"- An interrupted option needs a recovery contract before reuse.\n",
"\n",
"Execution: completed locally; constructed inputs are not deployment measurements.\n"
]
}
],
"source": [
"transfer_inputs = {'discount': 0.5,\n",
" 'deadline': 2,\n",
" 'start_time': 0,\n",
" 'options': [{'name': 'inspect',\n",
" 'initiation': True,\n",
" 'rewards': [0, 4],\n",
" 'terminated': True,\n",
" 'continuation_value': 8},\n",
" {'name': 'disabled',\n",
" 'initiation': False,\n",
" 'rewards': [9],\n",
" 'terminated': True,\n",
" 'continuation_value': 0}]}\n",
"transfer = analyze(chapter, transfer_inputs)\n",
"transfer['evidence_kind'] = 'constructed transfer example'\n",
"print(report_text(transfer))"
]
},
{
"cell_type": "markdown",
"id": "dad3f6b6584d",
"metadata": {},
"source": [
"## Apply the method to your inputs\n",
"\n",
"The example file below has the exact input shape the method accepts. Copy it to a new file, replace its values, then point `reader_file` at your copy. Run the cell again. Supplied inputs retain their stated provenance; the program cannot establish that they are representative observations.\n",
"\n",
"- **discount:** Gamma in [0,1].\n",
"- **deadline:** Nonnegative integer primitive-step deadline.\n",
"- **start_time:** Nonnegative elapsed time, same unit as deadline.\n",
"- **interrupt_after:** Optional nonnegative integer executed-step cap for each option.\n",
"- **options:** Each option has name,initiation:boolean,rewards:nonempty primitive reward list,terminated:boolean,continuation_value:number."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "15d1b4b7f2c3",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chapter 10: option-duration\n",
"Does a reusable skill finish before interruption or the deadline?\n",
"Evidence: supplied local inputs; provenance not independently verified\n",
"\n",
"Calculated quantities:\n",
"{\n",
" \"options\": [\n",
" {\n",
" \"name\": \"inspect\",\n",
" \"duration\": 2,\n",
" \"initiated\": true,\n",
" \"executed\": 2,\n",
" \"complete\": true,\n",
" \"value\": 4.0,\n",
" \"continuation_discount\": 0.25\n",
" },\n",
" {\n",
" \"name\": \"disabled\",\n",
" \"duration\": 1,\n",
" \"initiated\": false,\n",
" \"executed\": 0,\n",
" \"complete\": false,\n",
" \"value\": 0,\n",
" \"continuation_discount\": 0\n",
" }\n",
" ],\n",
" \"best_executed_value\": \"inspect\",\n",
" \"completed_options\": 1\n",
"}\n",
"\n",
"Interpretation:\n",
"An option receives continuation value only after its declared termination. A deadline or interruption can leave locally useful work without workflow completion.\n",
"\n",
"Assumptions:\n",
"- Rewards occur at primitive time offsets beginning at zero.\n",
"- Continuation discount uses the full option duration.\n",
"- Initiation and termination predicates are declared for this state.\n",
"\n",
"Limitations:\n",
"- The finite reward trace is not a learned option policy.\n",
"- An interrupted option needs a recovery contract before reuse.\n",
"\n",
"Execution: completed locally; constructed inputs are not deployment measurements.\n"
]
}
],
"source": [
"reader_file = LAB_ROOT / 'data/examples/ch10.json'\n",
"reader_inputs = json.loads(reader_file.read_text())\n",
"reader_report = analyze(chapter, reader_inputs)\n",
"print(report_text(reader_report))"
]
},
{
"cell_type": "markdown",
"id": "6ab74f6a58dd",
"metadata": {},
"source": [
"## Questions\n",
"\n",
"1. What is default continuation discount?\n",
"\n",
"2. What is interrupted review-release value?\n",
"\n",
"3. Why is transfer continuation contribution 2 rather than 4?\n",
"\n",
"Answers: [separate solutions](../solutions/ch10.md). Try the calculation before opening them."
]
},
{
"cell_type": "markdown",
"id": "44849a14851d",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"Options compress plans without erasing duration. Completed continuation is discounted by the number of primitive steps and is withheld when initiation, deadline, interruption, or termination prevents completion. The default favors review-release; interruption reverses that choice; the transfer case checks duration discounting. The result is a finite option-contract analysis, not a proof that a real skill runs reliably or resumes safely.\n",
"\n",
"Limits of this experiment:\n",
"\n",
"- This is a local calculation under declared inputs, not an empirical claim about a deployed agent.\n",
"- Read the returned assumptions and limitations before applying the numerical result.\n",
"\n",
"The assistant skill is [`maa-10-option-duration`](../skills/maa-10-option-duration/SKILL.md). It uses this notebook's tested computation and input contract."
]
},
{
"cell_type": "markdown",
"id": "9cc9f25ba898",
"metadata": {},
"source": [
"## Equations from the chapter\n",
"\n",
"These are the unchanged display equations and their explanations from the canonical chapter. They are a reference for the experiment, not a claim that every equation is numerically implemented by this one method."
]
},
{
"cell_type": "markdown",
"id": "7bbedb27c8a1",
"metadata": {},
"source": [
"### Equation 10.1\n",
"\n",
"\n",
"\n",
"Equation (10.1) separates where an option may start, how it acts while running, and how it stops.\n",
"\n",
"Start only in `\\mathcal{I}_o`; use `\\pi_o` for primitive choices; use `\\beta_o` to determine termination.\n",
"\n",
"LaTeX source, preserved for inspection:\n",
"\n",
"```latex\n",
"o=(\\mathcal{I}_o,\\pi_o,\\beta_o).\n",
"\\tag{10.1}\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "1a4342932925",
"metadata": {},
"source": [
"### Equation 10.2\n",
"\n",
"\n",
"\n",
"Equation (10.2) values rewards inside an option and value remaining after variable-duration completion.\n",
"\n",
"Discount internal primitive rewards, then discount continuation by actual primitive steps used.\n",
"\n",
"LaTeX source, preserved for inspection:\n",
"\n",
"```latex\n",
"Q^{\\mu}(x,o)=\\operatorname E_\\mu\\!\\left[\n",
"\\sum_{k=0}^{\\tau-1}\\gamma^k r_{t+k}\n",
"+\\gamma^\\tau V^\\mu(x_{t+\\tau})\\;\\middle|\\;x_t=x,o_t=o\n",
"\\right].\n",
"\\tag{10.2}\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "a41ddee6a2d8",
"metadata": {},
"source": [
"### Equation 10.3\n",
"\n",
"\n",
"\n",
"Equation (10.3) keeps completion delay separate from total tool cost in parallel workflow.\n",
"\n",
"Take longest dependency chain for time; add all required activity credits for cost.\n",
"\n",
"LaTeX source, preserved for inspection:\n",
"\n",
"```latex\n",
"T_{\\mathrm{release}}=\\max\\{2+3+1,\\;4\\}=6\\text{ minutes},\n",
"\\qquad C_{\\mathrm{tools}}=4+7+2+3=16\\text{ credits}.\n",
"\\tag{10.3}\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Mathematics of AI Agents",
"language": "python",
"name": "maa-lab"
},
"lab_chapter": 10,
"lab_execution": {
"code_cells": 6,
"created_utc": "2026-10-02T05:14:24.228361+00:00",
"elapsed_seconds": 8.845190958003514,
"method": "fresh process; new ipykernel InProcessKernelManager; cells submitted as Jupyter execute requests",
"network_transport_tested": false,
"python": "3.11.15",
"source_sha256": "9088a581db0ecff1dfaa2940a1f49e00cadc677c9fe39a426ca384110f88237d"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}