Empirical Retirement of Live Rules: Wiring the Sequential Probability Ratio Test into Real Conversion Streams to Promote, Demote and Retire Copy Rules: chapter answers
A business team often has dozens of copy, button and private-message rules on test at once. Common practice is to glance at the dashboard every week and promote whatever is significant, or, conversely, to believe that reliability means waiting for a full fixed sample. This book tests both defaults and wires Wald's sequential probability ratio test into a lifecycle state machine for rules, so that every rule passes through drafting, queueing, trial, promotion, watch, demotion and retirement under one discipline. All calculations use only the Python standard library. Conversion rates, traffic a…
DOI pending · monograph page goes live after publication · 5DL-B-263 · 中文版
- Chapter 1 · A Weekly Glance: How a Mediocre Copy Rule Gets Promoted by Mistake
Chapter 1 states and quantifies the claim: ten weekly two-sided looks give a false-alarm rate of 0.1934 and a year of looks 0.3235; in a queue of 40 rules peeking promotes 5.58 useless rules and the sequential test 0.73… - Chapter 2 · Wald's Two Straight Lines: Boundaries, the OC Function and the Mean Sample Size
Wald's sequential test writes evidence as a visitor-by-visitor sum, and its two boundaries are parallel lines in the count plane with slope 0.03291 and a gap of 22.97 conversions, which anyone can compare with the cumul… - Chapter 3 · Exact Recursions: the Long Tail of the Sample Size and the Cost of a Cap
Exact recursion adds three things to Wald's formulas. First, the true error rates are 0.0470 and 0.2000, both inside the bounds 0.0625 and 0.2105. Second, the mean sample size is 0.464 and 0.632 of the fixed sample at t… - Chapter 4 · Connecting Real Conversion Streams: Weekly Batches and Where the Baseline Comes From
Chapter 4 shows that weekly batches make the test more conservative, while re-estimating the baseline every week lifts false promotions to 0.1138. - Chapter 5 · The Rule Lifecycle as a State Machine: Promotion, Demotion and Retirement
Chapter 5 checks an eight-state rule machine exhaustively, watches for wear with Page's cumulative sum, and uses an absorbing chain to show that a worn rule stays live 8.54 weeks on average. - Chapter 6 · Many Rules in Parallel: Sharing Alpha, and How Many Promoted Rules Are Duds
Chapter 6 shows that splitting alpha among 100 rules lengthens verdicts on useless rules by a ratio of only 1.12. - Chapter 7 · Classical Sources East and West: Checking by Trial in the Han Feizi, and Wald's Sequential Analysis
"Certainty without checking is folly" from the Han Feizi becomes a concrete number in p71: promote without trial and 0.9000 of the promotion list is useless. Harnessing the horse and watching it to the end of the road i… - Chapter 8 · Closing the Loop: Wiring the Weekly Radar's Verdicts into the Rule State Machine
Chapter 8 wires the weekly radar's verdicts into the state machine and gives the distilled tool, the Rule Promotion and Demotion Judge, and a pre-registered A/A calibration: 100 A/A rules, with 11 or more promotions ove…
Illustrative model calculation, not a live quote