What does The Life of a Rule conclude, in numbers, about “Closing the Loop: Wiring the Weekly Radar's Verdicts into the Rule State Machine”?
Answer: Chapter 8 wires the weekly radar's verdicts into the state machine and gives the distilled tool, the Rule Promotion and Demotion Judge, and a pre-registered A/A calibration: 100 A/A rules, with 11 or more promotions overturning the book. Marked boxes separate published evidence, runnable Python calculations (20 programs), classical sources and the author's hypotheses.
Scope of the numbers: Marked boxes separate published evidence, runnable Python calculations (20 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 100, 11, 20
| A/A rules | reject if KEEPs >= | size@0.0625 | size@0.0332 | pow@0.10 | pow@0.15 |
| 20 | 4 | 0.0330 | 0.0038 | 0.1330 | 0.3523 |
| 40 | 6 | 0.0367 | 0.0019 | 0.2063 | 0.5675 |
| 60 | 8 | 0.0326 | 0.0008 | 0.2484 | 0.6953 |
| 80 | 10 | 0.0273 | 0.0003 | 0.2766 | 0.7789 |
| 100 | 11 | 0.0480 | 0.0005 | 0.4168 | 0.9006 |
| 120 | 13 | 0.0375 | 0.0002 | 0.4239 | 0.9254 |
| 150 | 15 | 0.0491 | 0.0002 | 0.5398 | 0.9717 |
| 200 | 19 | 0.0462 | 0.0000 | 0.6276 | 0.9918 |
| smallest listed n with power >= 0.80 against 0.15: 100 | |||||
| at 20 A/A slots and 5.83 weeks per null trial: 29.1 weeks |
Show the full program
# Chapter 8. p82: how many A/A calibration rules the pre-registration needs
# Model: an A/A rule sends traffic to a variant identical to control, so
# every KEEP it earns is a false promotion. The book's claim is that the
# weekly SPRT pipeline keeps the false-KEEP rate at or below Wald's
# bound 0.0625; its exact value at 600 visitors a week is 0.0332 (p41).
# The rival claim is that it behaves like weekly peeking, near 0.15 or
# more (p11, p12). For n A/A rules we pick the smallest critical count
# c with exact one-sided binomial size <= 0.05 under rate 0.0625, and
# report the power against 0.10 and 0.15. Exact binomial sums.
import math
R0, R_TRUE = 0.0625, 0.0332
def tail(n, c, p):
terms = [math.exp(math.lgamma(n + 1) - math.lgamma(j + 1)
- math.lgamma(n - j + 1) + j * math.log(p)
+ (n - j) * math.log(1 - p))
for j in range(c, n + 1)]
return math.fsum(terms)
def crit(n):
for c in range(n + 1):
if tail(n, c, R0) <= 0.05:
return c
return n + 1
print("A/A rules reject if KEEPs >= size@0.0625 size@0.0332"
" pow@0.10 pow@0.15")
first = None
for n in (20, 40, 60, 80, 100, 120, 150, 200):
c = crit(n)
pw = tail(n, c, 0.15)
if first is None and pw >= 0.8:
first = n
print("%9d %18d %11.4f %11.4f %8.4f %8.4f"
% (n, c, tail(n, c, R0), tail(n, c, R_TRUE), tail(n, c, 0.10),
pw))
print("smallest listed n with power >= 0.80 against 0.15: %d" % first)
weeks = first * 5.83 / 20
print("at 20 A/A slots and 5.83 weeks per null trial: %.1f weeks" % weeks)Single source and citation
Fan, Y. (2026). Empirical Retirement of Live Rules: Wiring the Sequential Probability Ratio Test into Real Conversion Streams to Promote, Demote and Retire Copy Rules (Version 1.0.0), Chapter 8: Closing the Loop: Wiring the Weekly Radar's Verdicts into the Rule State Machine. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-263 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026263,
author = {Fan, Yafu},
title = {Empirical Retirement of Live Rules: Wiring the Sequential Probability Ratio Test into Real Conversion Streams to Promote, Demote and Retire Copy Rules},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · A Weekly Glance: How a Mediocre Copy Rule Gets Promoted by Mistake
- Chapter 2 · Wald's Two Straight Lines: Boundaries, the OC Function and the Mean Sample Size
- Chapter 3 · Exact Recursions: the Long Tail of the Sample Size and the Cost of a Cap
- Chapter 4 · Connecting Real Conversion Streams: Weekly Batches and Where the Baseline Comes From
- Chapter 5 · The Rule Lifecycle as a State Machine: Promotion, Demotion and Retirement
- Chapter 6 · Many Rules in Parallel: Sharing Alpha, and How Many Promoted Rules Are Duds
- Chapter 7 · Classical Sources East and West: Checking by Trial in the Han Feizi, and Wald's Sequential Analysis