1Demonstration 1 of 4
The best average is excluded
Does the policy with the highest mean win?
The ceiling excludes a policy that breaches it in any draw, whatever its mean. Among those left, the one with the best worst case is recommended.
Three policies set the word-of-mouth strength: push 0.55, steady 0.35, hold 0.15. Each meets the same draws of market size between 700 and 1300. The ceiling is a bound on adopters in every draw.
Predict first. At a ceiling of 1100 and 25 draws, which policy is recommended although it has the lowest mean?
Choose an example
Constructed example: the chapter's three policies, 25 draws and seed 7, with other ceilings and draw counts defined for this reader, compared by the pack's compare and recommend functions.
Calculated values
- Push: mean, worst case
- 934.1, 722.4
- Push: draws above the ceiling
- 4 of 25
- Steady: mean, worst case
- 911.9, 705.5
- Steady: draws above the ceiling
- 4 of 25
- Hold: mean, worst case
- 533.6, 413.6
- Hold: draws above the ceiling
- 0 of 25
- Recommendation
- Hold
Push has the best mean, 934.1, but its highest draw 1285.6 - 1100 = 185.6 breaches the ceiling in 4 of 25 draws, so it is excluded. Recommended: Hold, worst case 413.6, mean 533.6. Mean given up: 934.1 - 533.6 = 400.5 adopters. Ranking is by worst case among the admissible policies.
Use the idea
Declare the ceiling and its reason before running the search, so an exclusion cannot be argued away afterwards.
Where the conclusion applies
Three named policies, one uncertain quantity, seed 7. The result changes with the number of draws, because fewer draws may miss the draws that breach.
Check your understanding: At a ceiling of 1300 with 25 draws, which policy is recommended?
Chapter 30 source: section "The policy with the best average". Demonstration C30-D01.
2Demonstration 2 of 4
Every draw, not the average
What changes when a ceiling is checked on the average instead of on every draw?
A mean can sit well below a ceiling that some draws exceed. Checking the average lets a policy pass that breaches in the worlds where the ceiling is reached.
The ceiling is a bound on adopters. Judged on every draw, one breach excludes a policy. Judged on the average, only the mean is compared with the ceiling.
Predict first. At a ceiling of 1100, which policy does the average-based check recommend?
Choose an example
Constructed example: the chapter's policies, draws and 1100 ceiling, with other ceilings defined for this reader, judged with the pack's Bound and recommend.
Calculated values
- Judged on
- every draw
- Push
- excluded
- Steady
- excluded
- Hold
- admissible
- Recommendation
- Hold
- First violation message
- adopters=1196 above 1100.0 (support capacity ceiling)
On every draw, push goes above 1100 in 4 of 25 draws (highest 1285.6 - 1100 = 185.6). Recommended: Hold, worst case 413.6, mean 533.6.
Use the idea
Write the constraint to be checked on every draw, because a capacity is exceeded on the days it is exceeded.
Where the conclusion applies
Twenty-five draws of one uncertain quantity. A ceiling above every draw cannot distinguish the two checks.
Check your understanding: At a ceiling of 900, does push pass the average-based check?
Chapter 30 source: section "Constraints are not preferences". Demonstration C30-D02.
3Demonstration 3 of 4
Same draws, every policy
Can two close policies be compared on different draws?
Different draws mean different markets, and the market moves adopters by more than the policies' gap. On shared draws the same markets meet both policies, so the gap belongs to the policies.
Push and steady differ in word-of-mouth strength, 0.55 and 0.35. Each is evaluated on market-size draws from its seed.
Predict first. With 5 draws each and a different seed for steady, can steady read ahead of push?
Choose an example
Constructed example: the chapter's push and steady policies on its market-size range, with separate seeds defined for this reader, run with the pack's evaluate function.
Calculated values
- Draws
- 25
- Draw design
- shared seed 7
- Push mean
- 934.1
- Steady mean
- 911.9
- Push minus steady
- 22.2
Push minus steady = 934.1 - 911.9 = 22.2. Push reads ahead. With shared draws the same 25 markets meet both policies, so the gap is the policies' alone.
Use the idea
Keep the seed in the run settings so a comparison can be repeated with the same draws.
Where the conclusion applies
Two policies, a few seeds chosen for this reader. Whether a given pair of seeds reverses the order is a property of those seeds, not a rate.
Check your understanding: On shared draws with 25 draws, which policy has the higher mean, and by how much?
Chapter 30 source: section "Same draws, every policy". Demonstration C30-D03.
4Demonstration 4 of 4
When no policy is admissible
What does the search return when every candidate breaks a constraint?
Both breach the ceiling, so the search returns nothing. Adding a lower-risk policy to the candidate set can restore an option; a ceiling below every draw of every policy cannot be met.
A candidate set is the policies somebody proposed. The ceiling is a bound on adopters, checked on all 25 draws.
Predict first. With only push and steady as candidates and a ceiling of 1100, does the search recommend either?
Choose an example
Constructed example: the chapter's policies and seed, with the ceilings and candidate sets defined for this reader, searched with the pack's recommend function.
Calculated values
- Candidates
- Push, Steady
- Ceiling
- 1100
- Push: draws above the ceiling
- 4 of 25
- Steady: draws above the ceiling
- 4 of 25
- Recommendation
- none: every policy violated a stated constraint
Highest draw minus the ceiling: Push 1285.6 - 1100 = 185.6; Steady 1254.6 - 1100 = 154.6. A positive result is a breach. Every policy violated a stated constraint, so the search returns no recommendation. The response is a decision: widen the candidate set, restate the constraint in the open, or report that no option is acceptable.
Use the idea
Treat an empty recommendation as a finding: widen the candidates, restate the constraint in the open, or report that no option is acceptable.
Where the conclusion applies
Policies are fixed settings and the ceiling is fixed. The failure to avoid is quietly relaxing the ceiling to get an answer.
Check your understanding: With all three candidates and a ceiling of 600, which policy is recommended?
Chapter 30 source: section "When no policy is admissible". Demonstration C30-D04.