关于「实证闭环:周度复盘雷达的多规则校正与证伪方案」,《伪发现率与文案规则检验》给出的定量结论是什么?
结论:第八章给周度复盘雷达加上多规则校正,e-BH 把伪发现率从 0.0896 降到 0.0023,并给出蒸馏小工具「伪发现率阶梯器」与预注册证伪方案。书中以标注框区分已发表的证据、可运行的 Python 计算(23 段程序)、原典与作者的假说。
数据口径:书中转化率、访客数与规则条数均为示意参数;全部计算由书中程序生成;本书属于非个人化经营建议。 本书用精确概率推翻这条常识:四十条全部无效的规则同时检验,平均每周冒出两条假冠军,至少一条的概率为 0.8715;模拟中每周最佳假冠军宣称的相对提升平均为 34.2 %,而真实提升为零。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0023, 0.0896
| A = 2.7726, B = -1.5581, 24 rules, 4 real | ||||
| layer | false/run | P(any false) | FDR | power |
| SPRT KEEP | 0.424 | 0.3540 | 0.0896 | 0.886 |
| e-BH | 0.009 | 0.0095 | 0.0023 | 0.792 |
| Ville+BH | 0.054 | 0.0515 | 0.0120 | 0.833 |
查看完整验算程序
# Chapter 8. p81: a weekly radar that runs one SPRT per copy rule, and
# the multiple-rule layer added on top. Model (illustrative): 24 rules,
# baseline p0 = 0.030 known, mde 20 % (p1 = 0.036), alpha 0.05, beta
# 0.20, Wald bounds A = ln(16), B = ln(0.2/0.95); 1800 visitors per
# rule per week, 8 weeks; four rules truly convert at 0.036, the rest
# at 0.030. Counts use the normal approximation to the binomial.
# Layer 1 (as is): a rule is KEEP once its log-likelihood ratio
# crosses A. Layer 2: at the week-8 review the likelihood ratio of
# each rule is an e-value; e-BH (Wang and Ramdas 2022) rejects the k
# largest with e_(k) >= m / (q k), valid under any dependence.
# Layer 3: Ville p-value 1 / max_t LR_t, then BH. 2000 runs, seed
# 20261113.
import math
import random
M, REAL, P0, MDE, ALPHA, BETA = 24, 4, 0.030, 0.20, 0.05, 0.20
NW, WEEKS, Q, RUNS, SEED = 1800, 8, 0.05, 2000, 20261113
P1 = P0 * (1 + MDE)
A = math.log((1 - BETA) / ALPHA)
B = math.log(BETA / (1 - ALPHA))
UP, DN = math.log(P1 / P0), math.log((1 - P1) / (1 - P0))
def rule_path(rng, p):
terms, top, keep, decided = [], 0.0, False, False
for _ in range(WEEKS):
k = round(NW * p + math.sqrt(NW * p * (1 - p)) * rng.gauss(0, 1))
k = max(0, k)
terms.append(k * UP + (NW - k) * DN)
llr = math.fsum(terms)
top = max(top, llr)
if not decided and llr >= A:
keep, decided = True, True
elif not decided and llr <= B:
decided = True
return keep, math.exp(llr), min(1.0, math.exp(-top))
def ebh(es, q):
m = len(es)
idx = sorted(range(m), key=lambda i: (-es[i], i))
kmax = 0
for k, i in enumerate(idx, 1):
if es[i] >= m / (q * k):
kmax = k
return set(idx[:kmax])
def bh(ps, q):
m = len(ps)
idx = sorted(range(m), key=lambda i: (ps[i], i))
kmax = 0
for k, i in enumerate(idx, 1):
if ps[i] <= k * q / m:
kmax = k
return set(idx[:kmax])
if __name__ == "__main__":
rng = random.Random(20261113)
stats = {n: ([], [], []) for n in ("SPRT KEEP", "e-BH", "Ville+BH")}
for _ in range(RUNS):
res = [rule_path(rng, P1 if i < REAL else P0)
for i in range(M)]
sets = {"SPRT KEEP": {i for i in range(M) if res[i][0]},
"e-BH": ebh([r[1] for r in res], Q),
"Ville+BH": bh([r[2] for r in res], Q)}
for n, s in sets.items():
v = len([i for i in s if i >= REAL])
stats[n][0].append(float(v))
stats[n][1].append(v / max(len(s), 1))
stats[n][2].append((len(s) - v) / REAL)
print("A = {:.4f}, B = {:.4f}, {} rules, {} real".format(A, B, M, REAL))
print("layer false/run P(any false) FDR power")
for n, (fv, fd, pw) in stats.items():
anyf = len([x for x in fv if x > 0]) / RUNS
print("{:10s} {:.3f} {:.4f} {:.4f} {:.3f}".format(
n, math.fsum(fv) / RUNS, anyf, math.fsum(fd) / RUNS,
math.fsum(pw) / RUNS))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《多重比较与伪发现率控制、同时检验数十条文案规则的本杰明尼霍赫伯格阶梯母典》第 8 章「实证闭环:周度复盘雷达的多规则校正与证伪方案」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-307 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026307,
author = {Fan, Yafu},
title = {Multiple Comparisons and False-Discovery-Rate Control: the Benjamini-Hochberg Step-Up for Testing Dozens of Copy Rules at Once},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}