关于「伪发现率与家族错误率:两个不同的目标」,《伪发现率与文案规则检验》给出的定量结论是什么?

结论:不校正时,出现假冠军的比例是 0.8722,与精确值 0.8715 相符,平均每周 1.9939 条。邦费罗尼与霍尔姆的比例都是 0.0469,在这种全部无效的情形下两者完全相同,因为霍尔姆的第一级门槛就是邦费罗尼门槛,第一级不过线就全部停止。BH 的比例是 0.0481,平均假发现 0.0529 条,只比邦费罗尼略高一点。BY 只有 0.0115,明显更保守。

数据口径:书中转化率、访客数与规则条数均为示意参数;全部计算由书中程序生成;本书属于非个人化经营建议。 本书用精确概率推翻这条常识:四十条全部无效的规则同时检验,平均每周冒出两条假冠军,至少一条的概率为 0.8715;模拟中每周最佳假冠军宣称的相对提升平均为 34.2 %,而真实提升为零。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0115, 0.0469, 0.0481, 0.0529, 0.8715, 0.8722

m = 40, all nulls true, 20000 weeks
ruleP(V>=1)=FWER=FDRmean V
none0.87221.9939
Bonf0.04690.0476
Holm0.04690.0476
BH0.04810.0529
BY0.01150.0116
exact for none: 0.8715; Bonferroni bound 0.05
查看完整验算程序
# Chapter 3. p31: when every rule is useless, FDR and FWER coincide.
# Model: m = 40 independent one-sided z-tests, all nulls true; 20000
# simulated weeks, seed 20261113. Level 0.05 for each procedure.
# With no true effect, any rejection is a false one and the false
# discovery proportion is 1, so FDR = P(V >= 1) = FWER.
import math
import random

M, Q, REPS, SEED = 40, 0.05, 20000, 20261113


def bh_count(ps, q):
    m, kmax = len(ps), 0
    for k, p in enumerate(sorted(ps), 1):
        if p <= k * q / m:
            kmax = k
    return kmax


def holm_count(ps, a):
    m = len(ps)
    for k, p in enumerate(sorted(ps)):
        if p > a / (m - k):
            return k
    return m


if __name__ == "__main__":
    rng = random.Random(20261113)
    hm = math.fsum(1.0 / j for j in range(1, M + 1))
    names = ("none", "Bonf", "Holm", "BH", "BY")
    hit = {n: [] for n in names}
    vol = {n: [] for n in names}
    for _ in range(REPS):
        ps = [0.5 * math.erfc(rng.gauss(0, 1) / math.sqrt(2))
              for _ in range(M)]
        v = {"none": len([p for p in ps if p < Q]),
             "Bonf": len([p for p in ps if p <= Q / M]),
             "Holm": holm_count(ps, Q),
             "BH": bh_count(ps, Q),
             "BY": bh_count(ps, Q / hm)}
        for n in names:
            hit[n].append(1.0 if v[n] else 0.0)
            vol[n].append(float(v[n]))
    print("m = 40, all nulls true, {} weeks".format(REPS))
    print("rule   P(V>=1)=FWER=FDR   mean V")
    for n in names:
        print("{:5s}  {:.4f}             {:.4f}".format(
            n, math.fsum(hit[n]) / REPS, math.fsum(vol[n]) / REPS))
    print("exact for none: {:.4f}; Bonferroni bound 0.05".format(
        1 - 0.95 ** M))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《多重比较与伪发现率控制、同时检验数十条文案规则的本杰明尼霍赫伯格阶梯母典》第 3 章「伪发现率与家族错误率:两个不同的目标」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-307 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026307,
  author    = {Fan, Yafu},
  title     = {Multiple Comparisons and False-Discovery-Rate Control: the Benjamini-Hochberg Step-Up for Testing Dozens of Copy Rules at Once},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论