{
"cells": [
{
"cell_type": "markdown",
"id": "d1dc8688b9e6",
"metadata": {},
"source": [
"# Chapter 23: When the Environment Gives Instructions: The Mathematics of Agent Security\n",
"\n",
"A retrieved document contains words that look like instructions. The controller needs the document as evidence, but that does not give the document authority over the workflow. The critical boundary is whether data can alter control or cause an effect outside the user's permission contract.\n",
"\n",
"This notebook replays local fictitious events. A small monitor checks action containment, authority provenance, and current-version review before publish. It also records attempted promotion of untrusted instructions into control. Task completion and security violations have separate counters, because an attack can coexist with a superficially successful release. No live attack is executed and no real tool boundary is enforced by this calculation.\n",
"\n",
"**Outcome:** Replay a fictitious attack trace against capability, provenance, and review checks.\n",
"\n",
"- Inspect the declared input contract\n",
"- Predict the hand-checkable case\n",
"- Run the shared computation\n",
"- Change the critical assumption\n",
"- Apply the method to the transfer data\n",
"\n",
"**Guided route:** Run the worked calculation, inspect its figure, change the stated assumption, and try the transfer case. Read the explanations beside each result before opening the answers.\n",
"\n",
"**Deeper route:** First read the mathematics and canonical equation reference. Audit the input contract, predict the changed result, then inspect the shared chapter implementation and solve the questions independently. Both routes use the same calculations and preserve the equations."
]
},
{
"cell_type": "markdown",
"id": "a7781f3ee7dc",
"metadata": {},
"source": [
"## Technical Requirements\n",
"\n",
"Python 3.11 or later, the complete laboratory folder, and the notebook dependencies listed in `requirements-notebooks.txt` (the launcher's **Install notebook tools** choice installs them; see START-HERE). Standard-library chapter commands also support Python 3.10. No API key, model account or network call is used by this experiment.\n",
"\n",
"Prior knowledge:\n",
"\n",
"- Python lists and dictionaries\n",
"- The mapped chapter and its notation"
]
},
{
"cell_type": "markdown",
"id": "1f435f9c4d23",
"metadata": {},
"source": [
"## The question and its mathematics\n",
"\n",
"Let C be the permitted capability set and source(a) the authority provenance of a requested action. An allowed action must lie in C and must not derive its authority from untrusted data. Publish also requires a valid review tied to the current document version.\n",
"\n",
"The monitor processes the trace in order, so a later review cannot retroactively authorize an earlier publish. A review for another version does not satisfy the current predicate. A supplied executed flag records whether the effect happened even when the monitor would reject it, exposing enforcement failure separately from rule evaluation.\n",
"\n",
"Security violations count both promoted instruction attempts and executed forbidden actions. Authorized task completion is true only when a publish event executes while satisfying capability, provenance, and review predicates. These counters can both be nonzero. The plot focuses on cumulative forbidden action executions; data-to-control promotion remains separately visible in metrics and event rows. This explicit scope prevents a narrow figure from pretending to summarize all security outcomes."
]
},
{
"cell_type": "markdown",
"id": "2e848773b5bb",
"metadata": {},
"source": [
"## A calculation you can run\n",
"\n",
"Provide capability names, current version, and an ordered event list. Data events state whether an instruction was attempted and promoted. Review events state validity and version. Action events state action name, provenance, executed status, and version for publish. Exact booleans and a closed provenance vocabulary prevent ambiguous inputs.\n",
"\n",
"The computation updates current review status and evaluates each action before recording its actual execution. It returns allowed and violated flags, monitor-denied count, blocked count (denied and not executed), total security violations, and authorized completion. The figure shows forbidden executions over event index. In the changed case, promote the injected instruction and execute the previously rejected request. Predict how total violations and the narrower plot differ. Keep the local attack-family boundary in the exported report.\n",
"\n",
"The next cell finds the bundle and imports the same computation used by the chapter skill. It does not change your system Python."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "42ebefc0e5c1",
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"import sys, json\n",
"LAB_ROOT = next((p for p in [Path.cwd(), *Path.cwd().parents] if (p / \"lab-manifest.json\").is_file()), None)\n",
"if LAB_ROOT is None:\n",
" raise RuntimeError(\"Open this notebook from the complete extracted laboratory folder.\")\n",
"sys.path.insert(0, str(LAB_ROOT / \"src\"))\n",
"from math_ai_agents.core import analyze, report_text\n",
"from math_ai_agents.plotting import figure_svg\n",
"from IPython.display import SVG, display\n"
]
},
{
"cell_type": "markdown",
"id": "7fffde8c0edb",
"metadata": {},
"source": [
"Set the declared inputs below. These are constructed teaching values, not measurements from a production agent. Change a value only after predicting what it should change."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d0364c02b280",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chapter 23: local-security-monitor\n",
"Did untrusted data cross into control or cause an unauthorized effect?\n",
"Evidence: constructed teaching example\n",
"\n",
"Calculated quantities:\n",
"{\n",
" \"monitor_denied_requests\": 1,\n",
" \"blocked_requests\": 1,\n",
" \"security_violations\": 0,\n",
" \"authorized_task_completion\": true,\n",
" \"event_count\": 4\n",
"}\n",
"\n",
"Interpretation:\n",
"The monitor checks capability containment, authority provenance, and current-version review before publish. Task completion is counted separately from violations.\n",
"\n",
"Assumptions:\n",
"- Attack family consists only of these local fictitious events.\n",
"- The supplied executed flag records what happened, even if the monitor rejected it.\n",
"- monitor_denied_requests counts actions the monitor would deny; blocked_requests counts only those denied actions that did not execute.\n",
"\n",
"Limitations:\n",
"- Checking a trace does not enforce a real tool boundary.\n",
"- Unrepresented attacks and omitted events are outside this monitor's coverage.\n",
"\n",
"Execution: completed locally; constructed inputs are not deployment measurements.\n"
]
}
],
"source": [
"chapter = 23\n",
"inputs = {'capabilities': ['read', 'publish'],\n",
" 'current_version': 'v2',\n",
" 'events': [{'kind': 'data', 'instruction_attempt': True, 'promoted_to_control': False},\n",
" {'kind': 'action',\n",
" 'action': 'publish',\n",
" 'authority_source': 'untrusted-data',\n",
" 'version': 'v2',\n",
" 'executed': False},\n",
" {'kind': 'review', 'valid': True, 'version': 'v2'},\n",
" {'kind': 'action',\n",
" 'action': 'publish',\n",
" 'authority_source': 'user',\n",
" 'version': 'v2',\n",
" 'executed': True}]}\n",
"report = analyze(chapter, inputs)\n",
"# This input was explicitly taken from the teaching fixture.\n",
"report['evidence_kind'] = 'constructed teaching example'\n",
"print(report_text(report))"
]
},
{
"cell_type": "markdown",
"id": "4ef1e2bafec3",
"metadata": {},
"source": [
"The default injected instruction remains data, and its untrusted publish request is rejected without execution. A valid current review then allows a user-authorized publish. The trace completes the task with zero security violations and one blocked request.\n",
"\n",
"The changed trace promotes the instruction and executes the untrusted publish before review. Those are two violations. Later authorized publish still completes the task. The plot rises once because it counts forbidden action executions; the total violation metric is two because it also counts data-to-control promotion. Completion has not canceled either violation. The monitor denies one request in this trace, but blocked_requests is zero because that request executed anyway.\n",
"\n",
"The plot below uses the calculated quantities. Read each panel's units before comparing its values."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "590f57e74048",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Matplotlib is building the font cache; this may take a moment.\n"
]
},
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display(SVG(figure_svg(report)))"
]
},
{
"cell_type": "markdown",
"id": "ae25c4323a26",
"metadata": {},
"source": [
"**Figure 23.L1:** Calculated chapter experiment. Each panel labels its input and output units; interpret it under the assumptions printed in the report."
]
},
{
"cell_type": "markdown",
"id": "eebcc4dfc4d6",
"metadata": {},
"source": [
"## Change the assumption\n",
"\n",
"A checker can identify a forbidden action without stopping it. The executed flag deliberately exposes that distinction. In a deployed system, enforcement belongs at the effect boundary with authenticated authority and durable records, not only in a language-model explanation.\n",
"\n",
"Monitor coverage is also local. An omitted event, an unsupported action family, or a compromised review source can escape this finite rule set. The notebook does not estimate worst-case security over all possible attacks. Its adversary consists of the supplied fictitious trace events.\n",
"\n",
"Version binding can fail through stale approval. The transfer fixture gives a valid review of A while current version is B. That review cannot authorize publish of B. Recent or correct-looking text is not sufficient provenance. Preserve review scope, capability containment, and task completion separately before claiming that an agent handled hostile information safely."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "a0b961d41d1c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chapter 23: local-security-monitor\n",
"Did untrusted data cross into control or cause an unauthorized effect?\n",
"Evidence: constructed changed-assumption example\n",
"\n",
"Calculated quantities:\n",
"{\n",
" \"monitor_denied_requests\": 1,\n",
" \"blocked_requests\": 0,\n",
" \"security_violations\": 2,\n",
" \"authorized_task_completion\": true,\n",
" \"event_count\": 4\n",
"}\n",
"\n",
"Interpretation:\n",
"The monitor checks capability containment, authority provenance, and current-version review before publish. Task completion is counted separately from violations.\n",
"\n",
"Assumptions:\n",
"- Attack family consists only of these local fictitious events.\n",
"- The supplied executed flag records what happened, even if the monitor rejected it.\n",
"- monitor_denied_requests counts actions the monitor would deny; blocked_requests counts only those denied actions that did not execute.\n",
"\n",
"Limitations:\n",
"- Checking a trace does not enforce a real tool boundary.\n",
"- Unrepresented attacks and omitted events are outside this monitor's coverage.\n",
"\n",
"Execution: completed locally; constructed inputs are not deployment measurements.\n"
]
},
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"changed_inputs = {'capabilities': ['read', 'publish'],\n",
" 'current_version': 'v2',\n",
" 'events': [{'kind': 'data', 'instruction_attempt': True, 'promoted_to_control': True},\n",
" {'kind': 'action',\n",
" 'action': 'publish',\n",
" 'authority_source': 'untrusted-data',\n",
" 'version': 'v2',\n",
" 'executed': True},\n",
" {'kind': 'review', 'valid': True, 'version': 'v2'},\n",
" {'kind': 'action',\n",
" 'action': 'publish',\n",
" 'authority_source': 'user',\n",
" 'version': 'v2',\n",
" 'executed': True}]}\n",
"changed = analyze(chapter, changed_inputs)\n",
"changed['evidence_kind'] = 'constructed changed-assumption example'\n",
"print(report_text(changed))\n",
"display(SVG(figure_svg(changed)))"
]
},
{
"cell_type": "markdown",
"id": "e736764a6b7a",
"metadata": {},
"source": [
"**Figure 23.L2:** The changed-assumption result. Compare the printed quantities and the stated assumptions with the first run. A different input need not imply a causal effect in a deployed agent."
]
},
{
"cell_type": "markdown",
"id": "ae5f695cf617",
"metadata": {},
"source": [
"## Try a new case\n",
"\n",
"The transfer trace has a stale review. Publish is monitor-rejected and not executed, so authorized completion is false and security violations remain zero. Refusal can therefore be secure without finishing the requested task. The useful next step is to obtain a valid review for the current version under an authorized source.\n",
"\n",
"For compatible local data, preserve both requested and executed actions. If actual execution is unknown, acquire effect evidence rather than marking it false. A trace report should identify the violated predicate and the enforcement boundary needed to prevent recurrence. It must not treat retrieved text or a model-generated justification as a new authority grant."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "148d680e4da8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chapter 23: local-security-monitor\n",
"Did untrusted data cross into control or cause an unauthorized effect?\n",
"Evidence: constructed transfer example\n",
"\n",
"Calculated quantities:\n",
"{\n",
" \"monitor_denied_requests\": 1,\n",
" \"blocked_requests\": 1,\n",
" \"security_violations\": 0,\n",
" \"authorized_task_completion\": false,\n",
" \"event_count\": 2\n",
"}\n",
"\n",
"Interpretation:\n",
"The monitor checks capability containment, authority provenance, and current-version review before publish. Task completion is counted separately from violations.\n",
"\n",
"Assumptions:\n",
"- Attack family consists only of these local fictitious events.\n",
"- The supplied executed flag records what happened, even if the monitor rejected it.\n",
"- monitor_denied_requests counts actions the monitor would deny; blocked_requests counts only those denied actions that did not execute.\n",
"\n",
"Limitations:\n",
"- Checking a trace does not enforce a real tool boundary.\n",
"- Unrepresented attacks and omitted events are outside this monitor's coverage.\n",
"\n",
"Execution: completed locally; constructed inputs are not deployment measurements.\n"
]
}
],
"source": [
"transfer_inputs = {'capabilities': ['read', 'publish'],\n",
" 'current_version': 'B',\n",
" 'events': [{'kind': 'review', 'valid': True, 'version': 'A'},\n",
" {'kind': 'action',\n",
" 'action': 'publish',\n",
" 'authority_source': 'system',\n",
" 'version': 'B',\n",
" 'executed': False}]}\n",
"transfer = analyze(chapter, transfer_inputs)\n",
"transfer['evidence_kind'] = 'constructed transfer example'\n",
"print(report_text(transfer))"
]
},
{
"cell_type": "markdown",
"id": "156bd040fb7e",
"metadata": {},
"source": [
"## Apply the method to your inputs\n",
"\n",
"The example file below has the exact input shape the method accepts. Copy it to a new file, replace its values, then point `reader_file` at your copy. Run the cell again. Supplied inputs retain their stated provenance; the program cannot establish that they are representative observations.\n",
"\n",
"- **capabilities:** List of permitted action names in the local contract.\n",
"- **current_version:** Current protected document version.\n",
"- **events:** Ordered data/review/action events. data:instruction_attempt,promoted_to_control booleans; review:valid boolean,version; action:action,authority_source=user/system/untrusted-data,version for publish,executed boolean."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "fcc2dab05f17",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Chapter 23: local-security-monitor\n",
"Did untrusted data cross into control or cause an unauthorized effect?\n",
"Evidence: supplied local inputs; provenance not independently verified\n",
"\n",
"Calculated quantities:\n",
"{\n",
" \"monitor_denied_requests\": 1,\n",
" \"blocked_requests\": 1,\n",
" \"security_violations\": 0,\n",
" \"authorized_task_completion\": false,\n",
" \"event_count\": 2\n",
"}\n",
"\n",
"Interpretation:\n",
"The monitor checks capability containment, authority provenance, and current-version review before publish. Task completion is counted separately from violations.\n",
"\n",
"Assumptions:\n",
"- Attack family consists only of these local fictitious events.\n",
"- The supplied executed flag records what happened, even if the monitor rejected it.\n",
"- monitor_denied_requests counts actions the monitor would deny; blocked_requests counts only those denied actions that did not execute.\n",
"\n",
"Limitations:\n",
"- Checking a trace does not enforce a real tool boundary.\n",
"- Unrepresented attacks and omitted events are outside this monitor's coverage.\n",
"\n",
"Execution: completed locally; constructed inputs are not deployment measurements.\n"
]
}
],
"source": [
"reader_file = LAB_ROOT / 'data/examples/ch23.json'\n",
"reader_inputs = json.loads(reader_file.read_text())\n",
"reader_report = analyze(chapter, reader_inputs)\n",
"print(report_text(reader_report))"
]
},
{
"cell_type": "markdown",
"id": "e9d2b36adc3f",
"metadata": {},
"source": [
"## Questions\n",
"\n",
"1. How many changed security violations occur?\n",
"\n",
"2. Why does the changed plot rise only once?\n",
"\n",
"3. Does a valid review of version A authorize publishing version B?\n",
"\n",
"Answers: [separate solutions](../solutions/ch23.md). Try the calculation before opening them."
]
},
{
"cell_type": "markdown",
"id": "33b77b3bd8ee",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"Untrusted data can be useful evidence without becoming control. The monitor checks local capabilities, provenance, and current-version review, then compares those rules with actual declared execution. The default completes securely, the changed trace completes with violations, and the transfer safely refuses stale approval. Security outcome and task outcome remain distinct. This finite replay diagnoses a supplied attack family without enforcing a real service or proving coverage of unknown adversaries.\n",
"\n",
"Limits of this experiment:\n",
"\n",
"- This is a local calculation under declared inputs, not an empirical claim about a deployed agent.\n",
"- Read the returned assumptions and limitations before applying the numerical result.\n",
"\n",
"The assistant skill is [`maa-23-local-security-monitor`](../skills/maa-23-local-security-monitor/SKILL.md). It uses this notebook's tested computation and input contract."
]
},
{
"cell_type": "markdown",
"id": "9cc9f25ba898",
"metadata": {},
"source": [
"## Equations from the chapter\n",
"\n",
"These are the unchanged display equations and their explanations from the canonical chapter. They are a reference for the experiment, not a claim that every equation is numerically implemented by this one method."
]
},
{
"cell_type": "markdown",
"id": "734040a518ee",
"metadata": {},
"source": [
"### Equation 23.1\n",
"\n",
"\n",
"\n",
"Bounds a delegated process's capability set by its parent's, so nothing reached through delegation can exceed the parent's own authority.\n",
"\n",
"A child inherits at most what it is given; delegation can narrow authority, never widen it.\n",
"\n",
"LaTeX source, preserved for inspection:\n",
"\n",
"```latex\n",
"\\mathcal C_{\\mathrm{child}}\\subseteq\\mathcal C_{\\mathrm{parent}}.\n",
"\\tag{23.1}\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "62a7d2b5ee21",
"metadata": {},
"source": [
"### Equation 23.2\n",
"\n",
"\n",
"\n",
"Advances the monitor's record from its previous state and one observed event, so authorization can be re-checked against current knowledge.\n",
"\n",
"Only real, observed events move `M_t` forward; a sentence claiming an event happened is not the event.\n",
"\n",
"LaTeX source, preserved for inspection:\n",
"\n",
"```latex\n",
"M_{t+1}=\\operatorname{Mon}(M_t,e_t).\n",
"\\tag{23.2}\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "8fe38f200a17",
"metadata": {},
"source": [
"### Equation 23.3\n",
"\n",
"\n",
"\n",
"Authorizes release only when the exact document, version, and recipient triple currently appears in the monitor's approval record.\n",
"\n",
"No partial match counts: the right document at the wrong version, or the right version to the wrong recipient, both fail the lookup.\n",
"\n",
"LaTeX source, preserved for inspection:\n",
"\n",
"```latex\n",
"\\operatorname{Publish}(\\delta,v,r,M_t)=1 \\iff (\\delta,v,r)\\in\\operatorname{Approved}(M_t).\n",
"\\tag{23.3}\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "ac25045f5203",
"metadata": {},
"source": [
"### Equation 23.4\n",
"\n",
"\n",
"\n",
"Defines exposure as the single worst violation chance across a declared attack family, not an average over the family.\n",
"\n",
"One member of `\\mathcal F` sets the value; a family with nine safe attacks and one dangerous one still reports the dangerous one's chance.\n",
"\n",
"LaTeX source, preserved for inspection:\n",
"\n",
"```latex\n",
"\\operatorname{Risk}(\\mathcal F)=\\max_{f\\in\\mathcal F}\\Pr[\\text{violation}\\mid f].\n",
"\\tag{23.4}\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Mathematics of AI Agents",
"language": "python",
"name": "maa-lab"
},
"lab_chapter": 23,
"lab_execution": {
"code_cells": 6,
"created_utc": "2026-10-02T05:16:23.611312+00:00",
"elapsed_seconds": 8.829025083919987,
"method": "fresh process; new ipykernel InProcessKernelManager; cells submitted as Jupyter execute requests",
"network_transport_tested": false,
"python": "3.11.15",
"source_sha256": "46dbb4e2ef8f3098fa0143012e30fed968a4e23f4b77b410ffd10920c73fe11a"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}