What does The Life of a Rule conclude, in numbers, about “A Weekly Glance: How a Mediocre Copy Rule Gets Promoted by Mistake”?
Answer: Chapter 1 states and quantifies the claim: ten weekly two-sided looks give a false-alarm rate of 0.1934 and a year of looks 0.3235; in a queue of 40 rules peeking promotes 5.58 useless rules and the sequential test 0.73. The preface corrects the blueprint: one in five is passed only at the eleventh look; "about half" holds only near the design points, where the exact values are 0.464 and 0.632 of the fixed sample, rising to 0.748 for middling rules; and optimality was proved by Wald and Wolfowitz in 1948.
Scope of the numbers: Marked boxes separate published evidence, runnable Python calculations (20 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.1934, 0.3235, 10, 11
| repeated two-sided z-tests at nominal 0.05, one look per week | |||
| grid width 0.02, looks up to 52 | |||
| looks | P(false win by then) | x nominal | false wins in 30 nulls |
| 1 | 0.0500 | 1.00 | 1.50 |
| 2 | 0.0831 | 1.66 | 2.49 |
| 3 | 0.1073 | 2.15 | 3.22 |
| 5 | 0.1417 | 2.83 | 4.25 |
| 8 | 0.1763 | 3.53 | 5.29 |
| 10 | 0.1934 | 3.87 | 5.80 |
| 13 | 0.2138 | 4.28 | 6.41 |
| 20 | 0.2479 | 4.96 | 7.44 |
| 26 | 0.2688 | 5.38 | 8.06 |
| 39 | 0.3009 | 6.02 | 9.03 |
| 52 | 0.3235 | 6.47 | 9.71 |
| first look count with P(false win) >= 0.20: 11 |
Show the full program
# Chapter 1. p11: weekly looks at a fixed-sample test inflate false wins
# Model: the classical repeated significance test of Armitage, McPherson
# and Rowe (1969). Each weekly batch adds one N(0,1) increment to the
# score S_k; at look k the rule is promoted if |S_k| >= 1.96 * sqrt(k)
# (nominal two-sided 0.05). We push the density of S_k forward on a
# grid of cells of width H, using exact normal cell masses (erf), and
# remove the mass that crosses at each look (a cell cut by the
# boundary is split in proportion). Nothing is random here. Grid width
# H = 0.02 is a numerical choice; results are good to about three
# decimals. The pipeline line multiplies by 30 null copy rules
# (an illustrative backlog, not data).
import math
H = 0.02
Z = 1.96
LOOKS = 52
SHOW = (1, 2, 3, 5, 8, 10, 13, 20, 26, 39, 52)
NULL_RULES = 30
def phi_cdf(x):
return 0.5 * (1.0 + math.erf(x / math.sqrt(2.0)))
def kernel(width, reach=8.0):
d = int(reach / width) + 1
ker = {}
for j in range(-d, d + 1):
lo = (j - 0.5) * width
ker[j] = phi_cdf(lo + width) - phi_cdf(lo)
return d, ker
def run():
d, ker = kernel(H)
mass = {0: 1.0} # cell index -> probability, start at S_0 = 0
crossed = []
for k in range(1, LOOKS + 1):
lim = Z * math.sqrt(k)
top = int(lim / H) + d + 2
new = {}
for j in range(-top, top + 1):
terms = [m * ker[j - i] for i, m in mass.items()
if -d <= j - i <= d]
if terms:
new[j] = math.fsum(terms)
keep, gone = {}, []
for j in sorted(new):
lo = max(j * H - H / 2, -lim)
hi = min(j * H + H / 2, lim)
inside = max(0.0, hi - lo) / H
if inside > 0:
keep[j] = new[j] * inside
gone.append(new[j] * (1.0 - inside))
crossed.append(math.fsum(gone))
mass = keep
return crossed
crossed = run()
print("repeated two-sided z-tests at nominal 0.05, one look per week")
print("grid width %.2f, looks up to %d" % (H, LOOKS))
print("looks P(false win by then) x nominal false wins in %d nulls"
% NULL_RULES)
for k in SHOW:
a = math.fsum(crossed[:k])
print("%5d %20.4f %9.2f %21.2f" % (k, a, a / 0.05, a * NULL_RULES))
first = next(k for k in range(1, LOOKS + 1)
if math.fsum(crossed[:k]) >= 0.2)
print("first look count with P(false win) >= 0.20: %d" % first)Single source and citation
Fan, Y. (2026). Empirical Retirement of Live Rules: Wiring the Sequential Probability Ratio Test into Real Conversion Streams to Promote, Demote and Retire Copy Rules (Version 1.0.0), Chapter 1: A Weekly Glance: How a Mediocre Copy Rule Gets Promoted by Mistake. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-263 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026263,
author = {Fan, Yafu},
title = {Empirical Retirement of Live Rules: Wiring the Sequential Probability Ratio Test into Real Conversion Streams to Promote, Demote and Retire Copy Rules},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 2 · Wald's Two Straight Lines: Boundaries, the OC Function and the Mean Sample Size
- Chapter 3 · Exact Recursions: the Long Tail of the Sample Size and the Cost of a Cap
- Chapter 4 · Connecting Real Conversion Streams: Weekly Batches and Where the Baseline Comes From
- Chapter 5 · The Rule Lifecycle as a State Machine: Promotion, Demotion and Retirement
- Chapter 6 · Many Rules in Parallel: Sharing Alpha, and How Many Promoted Rules Are Duds
- Chapter 7 · Classical Sources East and West: Checking by Trial in the Han Feizi, and Wald's Sequential Analysis
- Chapter 8 · Closing the Loop: Wiring the Weekly Radar's Verdicts into the Rule State Machine