关于「样本量:每条规则要多少访客,共享对照怎么分流」,《伪发现率与文案规则检验》给出的定量结论是什么?

结论:第六章把校正换算成每组访客数:不校正 10,960 名,邦费罗尼 26,476 名,BH 在真规则占两成时约 18,555 名。

数据口径:书中转化率、访客数与规则条数均为示意参数;全部计算由书中程序生成;本书属于非个人化经营建议。 本书用精确概率推翻这条常识:四十条全部无效的规则同时检验,平均每周冒出两条假冠军,至少一条的概率为 0.8715;模拟中每周最佳假冠军宣称的相对提升平均为 34.2 %,而真实提升为零。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 18555, 26476

raw level 0.05:n per arm = 10,958
Bonferroni 0.05/40:n per arm = 26,476
pi1n per armBH thresholdequiv. mvs Bonf
0.0524,3210.00210023.80.919
0.1021,4410.00418911.90.810
0.2018,5550.0083336.00.701
0.3016,8690.0124364.00.637
0.5014,7510.0205132.40.557
查看完整验算程序
# Chapter 6. p62: the threshold BH settles on, and the n it implies.
# Model: many rules, a share pi1 real with the same lift; at n per arm
# a real rule's z-drift is d(n) = (p1 - p0) / sqrt((p0 q0 + p1 q1) / n)
# and its power at threshold t is G(t) = 1 - Phi(z_(1-t) - d). In the
# large-m limit BH rejects p <= t*, the largest t with
#   t = q (pi0 t + pi1 G(t)),
# found by iterating from t = q. We then search the smallest n with
# G(t*) >= 0.80. p0 = 0.030, lift 20 %, q = 0.05, m = 40 for
# Bonferroni. All parameters illustrative.
from statistics import NormalDist
import math

P0, LIFT, Q, M, TARGET = 0.030, 0.20, 0.05, 40, 0.80
ND = NormalDist()
P1 = P0 * (1 + LIFT)


def drift(n):
    return (P1 - P0) / math.sqrt((P0 * (1 - P0) + P1 * (1 - P1)) / n)


def power(t, n):
    return 1 - ND.cdf(ND.inv_cdf(1 - t) - drift(n))


def bh_threshold(pi1, n):
    t = Q
    for _ in range(2000):
        nxt = Q * ((1 - pi1) * t + pi1 * power(t, n))
        if abs(nxt - t) < 1e-15:
            break
        t = nxt
    return t


def smallest_n(fn):
    lo, hi = 10, 10 ** 7
    while lo < hi:
        mid = (lo + hi) // 2
        if fn(mid) >= TARGET:
            hi = mid
        else:
            lo = mid + 1
    return lo


if __name__ == "__main__":
    n_raw = smallest_n(lambda n: power(Q, n))
    n_bon = smallest_n(lambda n: power(Q / M, n))
    print("raw level 0.05:        n per arm = {:,}".format(n_raw))
    print("Bonferroni 0.05/40:    n per arm = {:,}".format(n_bon))
    print(" pi1   n per arm  BH threshold  equiv. m  vs Bonf")
    for pi1 in (0.05, 0.1, 0.2, 0.3, 0.5):
        n = smallest_n(lambda k: power(bh_threshold(pi1, k), k))
        t = bh_threshold(pi1, n)
        print(" {:.2f}  {:>9,}  {:.6f}      {:5.1f}     {:.3f}".format(
            pi1, n, t, Q / t, n / n_bon))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《多重比较与伪发现率控制、同时检验数十条文案规则的本杰明尼霍赫伯格阶梯母典》第 6 章「样本量:每条规则要多少访客,共享对照怎么分流」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-307 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026307,
  author    = {Fan, Yafu},
  title     = {Multiple Comparisons and False-Discovery-Rate Control: the Benjamini-Hochberg Step-Up for Testing Dozens of Copy Rules at Once},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论