关于「功效:真效应比例、效应大小与规则条数的扫描」,《伪发现率与文案规则检验》给出的定量结论是什么?
结论:邦费罗尼一列几乎不动,始终在 0.298 到 0.309 之间,因为它的门槛只取决于规则条数,与有多少真规则无关。霍尔姆略有上升,从 0.309 升到 0.325:真规则越多,前几级被拒绝后门槛放宽得越多。BH 的功效随真规则比例上升最快:占 0.05 时是 0.347,占 0.20 时是 0.483,占 0.50 时是 0.647,占 0.70 时是 0.706。真规则多,阶梯上过线的位置就靠后,每一级的门槛也就更宽。与此同时,BH 的伪发现率始终在 0.05 以下,从 0.0460 降到 0.0145,与上一章说的「无效比例乘以 q」一致。
数据口径:书中转化率、访客数与规则条数均为示意参数;全部计算由书中程序生成;本书属于非个人化经营建议。 本书用精确概率推翻这条常识:四十条全部无效的规则同时检验,平均每周冒出两条假冠军,至少一条的概率为 0.8715;模拟中每周最佳假冠军宣称的相对提升平均为 34.2 %,而真实提升为零。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0145, 0.0460, 0.05, 0.20, 0.298, 0.309
| pi1 m1 raw Bonf Holm BH BY FDR BH FDR raw | |
| 0.05 | 2 0.801 0.309 0.309 0.347 0.198 0.0460 0.4777 |
| 0.10 | 4 0.807 0.308 0.310 0.402 0.213 0.0440 0.3243 |
| 0.20 | 8 0.803 0.301 0.306 0.483 0.257 0.0395 0.1850 |
| 0.30 12 | 0.803 0.301 0.311 0.555 0.300 0.0361 0.1188 |
| 0.50 20 | 0.803 0.300 0.318 0.647 0.369 0.0254 0.0564 |
| 0.70 28 | 0.804 0.298 0.325 0.706 0.424 0.0145 0.0246 |
查看完整验算程序
# Chapter 4. p41: power as the share of real rules grows.
# Model: m = 40 independent one-sided z-tests; a share pi1 of rules
# has z-mean 2.5, the rest are null. Average power = expected share
# of real rules found. Level 0.05 for all; 5000 weeks per row, seed
# 20261113. Parameters illustrative.
import math
import random
M, Q, MU, REPS, SEED = 40, 0.05, 2.5, 5000, 20261113
def bh_k(sp, q):
m, kmax = len(sp), 0
for k, p in enumerate(sp, 1):
if p <= k * q / m:
kmax = k
return kmax
def holm_k(sp, a):
m = len(sp)
for k, p in enumerate(sp):
if p > a / (m - k):
return k
return m
def week(rng, m1):
out = []
for i in range(M):
z = rng.gauss(MU if i < m1 else 0.0, 1.0)
out.append((0.5 * math.erfc(z / math.sqrt(2)), i))
out.sort()
return out
if __name__ == "__main__":
rng = random.Random(20261113)
hm = math.fsum(1.0 / j for j in range(1, M + 1))
print(" pi1 m1 raw Bonf Holm BH BY FDR BH FDR raw")
for m1 in (2, 4, 8, 12, 20, 28):
pw = {k: [] for k in ("raw", "bon", "holm", "bh", "by")}
fdr_bh, fdr_raw = [], []
for _ in range(REPS):
w = week(rng, m1)
sp = [p for p, _ in w]
ks = {"raw": len([p for p in sp if p < Q]),
"bon": len([p for p in sp if p <= Q / M]),
"holm": holm_k(sp, Q), "bh": bh_k(sp, Q),
"by": bh_k(sp, Q / hm)}
for name, k in ks.items():
real = len([i for _, i in w[:k] if i < m1])
pw[name].append(real / m1)
if name == "bh":
fdr_bh.append((k - real) / max(k, 1))
if name == "raw":
fdr_raw.append((k - real) / max(k, 1))
cells = " ".join("{:.3f}".format(math.fsum(pw[k]) / REPS)
for k in ("raw", "bon", "holm", "bh", "by"))
print(" {:.2f} {:2d} {} {:.4f} {:.4f}".format(
m1 / M, m1, cells, math.fsum(fdr_bh) / REPS,
math.fsum(fdr_raw) / REPS))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《多重比较与伪发现率控制、同时检验数十条文案规则的本杰明尼霍赫伯格阶梯母典》第 4 章「功效:真效应比例、效应大小与规则条数的扫描」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-307 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026307,
author = {Fan, Yafu},
title = {Multiple Comparisons and False-Discovery-Rate Control: the Benjamini-Hochberg Step-Up for Testing Dozens of Copy Rules at Once},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}