What does The Life of a Rule conclude, in numbers, about “A Weekly Glance: How a Mediocre Copy Rule Gets Promoted by Mistake”?

Answer: Chapter 1 states and quantifies the claim: ten weekly two-sided looks give a false-alarm rate of 0.1934 and a year of looks 0.3235; in a queue of 40 rules peeking promotes 5.58 useless rules and the sequential test 0.73. The preface corrects the blueprint: one in five is passed only at the eleventh look; "about half" holds only near the design points, where the exact values are 0.464 and 0.632 of the fixed sample, rising to 0.748 for middling rules; and optimality was proved by Wald and Wolfowitz in 1948.

Scope of the numbers: Marked boxes separate published evidence, runnable Python calculations (20 programs), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.1934, 0.3235, 10, 11

repeated two-sided z-tests at nominal 0.05, one look per week
grid width 0.02, looks up to 52
looksP(false win by then)x nominalfalse wins in 30 nulls
10.05001.001.50
20.08311.662.49
30.10732.153.22
50.14172.834.25
80.17633.535.29
100.19343.875.80
130.21384.286.41
200.24794.967.44
260.26885.388.06
390.30096.029.03
520.32356.479.71
first look count with P(false win) >= 0.20: 11
Show the full program
# Chapter 1. p11: weekly looks at a fixed-sample test inflate false wins
# Model: the classical repeated significance test of Armitage, McPherson
# and Rowe (1969). Each weekly batch adds one N(0,1) increment to the
# score S_k; at look k the rule is promoted if |S_k| >= 1.96 * sqrt(k)
# (nominal two-sided 0.05). We push the density of S_k forward on a
# grid of cells of width H, using exact normal cell masses (erf), and
# remove the mass that crosses at each look (a cell cut by the
# boundary is split in proportion). Nothing is random here. Grid width
# H = 0.02 is a numerical choice; results are good to about three
# decimals. The pipeline line multiplies by 30 null copy rules
# (an illustrative backlog, not data).
import math

H = 0.02
Z = 1.96
LOOKS = 52
SHOW = (1, 2, 3, 5, 8, 10, 13, 20, 26, 39, 52)
NULL_RULES = 30


def phi_cdf(x):
    return 0.5 * (1.0 + math.erf(x / math.sqrt(2.0)))


def kernel(width, reach=8.0):
    d = int(reach / width) + 1
    ker = {}
    for j in range(-d, d + 1):
        lo = (j - 0.5) * width
        ker[j] = phi_cdf(lo + width) - phi_cdf(lo)
    return d, ker


def run():
    d, ker = kernel(H)
    mass = {0: 1.0}          # cell index -> probability, start at S_0 = 0
    crossed = []
    for k in range(1, LOOKS + 1):
        lim = Z * math.sqrt(k)
        top = int(lim / H) + d + 2
        new = {}
        for j in range(-top, top + 1):
            terms = [m * ker[j - i] for i, m in mass.items()
                     if -d <= j - i <= d]
            if terms:
                new[j] = math.fsum(terms)
        keep, gone = {}, []
        for j in sorted(new):
            lo = max(j * H - H / 2, -lim)
            hi = min(j * H + H / 2, lim)
            inside = max(0.0, hi - lo) / H
            if inside > 0:
                keep[j] = new[j] * inside
            gone.append(new[j] * (1.0 - inside))
        crossed.append(math.fsum(gone))
        mass = keep
    return crossed


crossed = run()
print("repeated two-sided z-tests at nominal 0.05, one look per week")
print("grid width %.2f, looks up to %d" % (H, LOOKS))
print("looks  P(false win by then)  x nominal  false wins in %d nulls"
      % NULL_RULES)
for k in SHOW:
    a = math.fsum(crossed[:k])
    print("%5d  %20.4f  %9.2f  %21.2f" % (k, a, a / 0.05, a * NULL_RULES))
first = next(k for k in range(1, LOOKS + 1)
             if math.fsum(crossed[:k]) >= 0.2)
print("first look count with P(false win) >= 0.20: %d" % first)

Single source and citation

Fan, Y. (2026). Empirical Retirement of Live Rules: Wiring the Sequential Probability Ratio Test into Real Conversion Streams to Promote, Demote and Retire Copy Rules (Version 1.0.0), Chapter 1: A Weekly Glance: How a Mediocre Copy Rule Gets Promoted by Mistake. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-263 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026263,
  author    = {Fan, Yafu},
  title     = {Empirical Retirement of Live Rules: Wiring the Sequential Probability Ratio Test into Real Conversion Streams to Promote, Demote and Retire Copy Rules},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book