关于「伪发现率与家族错误率:两个不同的目标」,《伪发现率与文案规则检验》给出的定量结论是什么?
结论:不校正时,出现假冠军的比例是 0.8722,与精确值 0.8715 相符,平均每周 1.9939 条。邦费罗尼与霍尔姆的比例都是 0.0469,在这种全部无效的情形下两者完全相同,因为霍尔姆的第一级门槛就是邦费罗尼门槛,第一级不过线就全部停止。BH 的比例是 0.0481,平均假发现 0.0529 条,只比邦费罗尼略高一点。BY 只有 0.0115,明显更保守。
数据口径:书中转化率、访客数与规则条数均为示意参数;全部计算由书中程序生成;本书属于非个人化经营建议。 本书用精确概率推翻这条常识:四十条全部无效的规则同时检验,平均每周冒出两条假冠军,至少一条的概率为 0.8715;模拟中每周最佳假冠军宣称的相对提升平均为 34.2 %,而真实提升为零。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0115, 0.0469, 0.0481, 0.0529, 0.8715, 0.8722
| m = 40, all nulls true, 20000 weeks | ||
| rule | P(V>=1)=FWER=FDR | mean V |
| none | 0.8722 | 1.9939 |
| Bonf | 0.0469 | 0.0476 |
| Holm | 0.0469 | 0.0476 |
| BH | 0.0481 | 0.0529 |
| BY | 0.0115 | 0.0116 |
| exact for none: 0.8715; Bonferroni bound 0.05 |
查看完整验算程序
# Chapter 3. p31: when every rule is useless, FDR and FWER coincide.
# Model: m = 40 independent one-sided z-tests, all nulls true; 20000
# simulated weeks, seed 20261113. Level 0.05 for each procedure.
# With no true effect, any rejection is a false one and the false
# discovery proportion is 1, so FDR = P(V >= 1) = FWER.
import math
import random
M, Q, REPS, SEED = 40, 0.05, 20000, 20261113
def bh_count(ps, q):
m, kmax = len(ps), 0
for k, p in enumerate(sorted(ps), 1):
if p <= k * q / m:
kmax = k
return kmax
def holm_count(ps, a):
m = len(ps)
for k, p in enumerate(sorted(ps)):
if p > a / (m - k):
return k
return m
if __name__ == "__main__":
rng = random.Random(20261113)
hm = math.fsum(1.0 / j for j in range(1, M + 1))
names = ("none", "Bonf", "Holm", "BH", "BY")
hit = {n: [] for n in names}
vol = {n: [] for n in names}
for _ in range(REPS):
ps = [0.5 * math.erfc(rng.gauss(0, 1) / math.sqrt(2))
for _ in range(M)]
v = {"none": len([p for p in ps if p < Q]),
"Bonf": len([p for p in ps if p <= Q / M]),
"Holm": holm_count(ps, Q),
"BH": bh_count(ps, Q),
"BY": bh_count(ps, Q / hm)}
for n in names:
hit[n].append(1.0 if v[n] else 0.0)
vol[n].append(float(v[n]))
print("m = 40, all nulls true, {} weeks".format(REPS))
print("rule P(V>=1)=FWER=FDR mean V")
for n in names:
print("{:5s} {:.4f} {:.4f}".format(
n, math.fsum(hit[n]) / REPS, math.fsum(vol[n]) / REPS))
print("exact for none: {:.4f}; Bonferroni bound 0.05".format(
1 - 0.95 ** M))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《多重比较与伪发现率控制、同时检验数十条文案规则的本杰明尼霍赫伯格阶梯母典》第 3 章「伪发现率与家族错误率:两个不同的目标」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-307 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026307,
author = {Fan, Yafu},
title = {Multiple Comparisons and False-Discovery-Rate Control: the Benjamini-Hochberg Step-Up for Testing Dozens of Copy Rules at Once},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}