关于「功效:真效应比例、效应大小与规则条数的扫描」,《伪发现率与文案规则检验》给出的定量结论是什么?

结论:邦费罗尼一列几乎不动,始终在 0.298 到 0.309 之间,因为它的门槛只取决于规则条数,与有多少真规则无关。霍尔姆略有上升,从 0.309 升到 0.325:真规则越多,前几级被拒绝后门槛放宽得越多。BH 的功效随真规则比例上升最快:占 0.05 时是 0.347,占 0.20 时是 0.483,占 0.50 时是 0.647,占 0.70 时是 0.706。真规则多,阶梯上过线的位置就靠后,每一级的门槛也就更宽。与此同时,BH 的伪发现率始终在 0.05 以下,从 0.0460 降到 0.0145,与上一章说的「无效比例乘以 q」一致。

数据口径:书中转化率、访客数与规则条数均为示意参数;全部计算由书中程序生成;本书属于非个人化经营建议。 本书用精确概率推翻这条常识:四十条全部无效的规则同时检验,平均每周冒出两条假冠军,至少一条的概率为 0.8715;模拟中每周最佳假冠军宣称的相对提升平均为 34.2 %,而真实提升为零。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0145, 0.0460, 0.05, 0.20, 0.298, 0.309

pi1 m1 raw Bonf Holm BH BY FDR BH FDR raw
0.052 0.801 0.309 0.309 0.347 0.198 0.0460 0.4777
0.104 0.807 0.308 0.310 0.402 0.213 0.0440 0.3243
0.208 0.803 0.301 0.306 0.483 0.257 0.0395 0.1850
0.30 120.803 0.301 0.311 0.555 0.300 0.0361 0.1188
0.50 200.803 0.300 0.318 0.647 0.369 0.0254 0.0564
0.70 280.804 0.298 0.325 0.706 0.424 0.0145 0.0246
查看完整验算程序
# Chapter 4. p41: power as the share of real rules grows.
# Model: m = 40 independent one-sided z-tests; a share pi1 of rules
# has z-mean 2.5, the rest are null. Average power = expected share
# of real rules found. Level 0.05 for all; 5000 weeks per row, seed
# 20261113. Parameters illustrative.
import math
import random

M, Q, MU, REPS, SEED = 40, 0.05, 2.5, 5000, 20261113


def bh_k(sp, q):
    m, kmax = len(sp), 0
    for k, p in enumerate(sp, 1):
        if p <= k * q / m:
            kmax = k
    return kmax


def holm_k(sp, a):
    m = len(sp)
    for k, p in enumerate(sp):
        if p > a / (m - k):
            return k
    return m


def week(rng, m1):
    out = []
    for i in range(M):
        z = rng.gauss(MU if i < m1 else 0.0, 1.0)
        out.append((0.5 * math.erfc(z / math.sqrt(2)), i))
    out.sort()
    return out


if __name__ == "__main__":
    rng = random.Random(20261113)
    hm = math.fsum(1.0 / j for j in range(1, M + 1))
    print(" pi1  m1   raw    Bonf   Holm   BH     BY     FDR BH  FDR raw")
    for m1 in (2, 4, 8, 12, 20, 28):
        pw = {k: [] for k in ("raw", "bon", "holm", "bh", "by")}
        fdr_bh, fdr_raw = [], []
        for _ in range(REPS):
            w = week(rng, m1)
            sp = [p for p, _ in w]
            ks = {"raw": len([p for p in sp if p < Q]),
                  "bon": len([p for p in sp if p <= Q / M]),
                  "holm": holm_k(sp, Q), "bh": bh_k(sp, Q),
                  "by": bh_k(sp, Q / hm)}
            for name, k in ks.items():
                real = len([i for _, i in w[:k] if i < m1])
                pw[name].append(real / m1)
                if name == "bh":
                    fdr_bh.append((k - real) / max(k, 1))
                if name == "raw":
                    fdr_raw.append((k - real) / max(k, 1))
        cells = " ".join("{:.3f}".format(math.fsum(pw[k]) / REPS)
                         for k in ("raw", "bon", "holm", "bh", "by"))
        print(" {:.2f} {:2d}   {}  {:.4f}  {:.4f}".format(
            m1 / M, m1, cells, math.fsum(fdr_bh) / REPS,
            math.fsum(fdr_raw) / REPS))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《多重比较与伪发现率控制、同时检验数十条文案规则的本杰明尼霍赫伯格阶梯母典》第 4 章「功效:真效应比例、效应大小与规则条数的扫描」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-307 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026307,
  author    = {Fan, Yafu},
  title     = {Multiple Comparisons and False-Discovery-Rate Control: the Benjamini-Hochberg Step-Up for Testing Dozens of Copy Rules at Once},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论