关于「实证闭环:周度复盘雷达的多规则校正与证伪方案」,《伪发现率与文案规则检验》给出的定量结论是什么?

结论:第八章给周度复盘雷达加上多规则校正,e-BH 把伪发现率从 0.0896 降到 0.0023,并给出蒸馏小工具「伪发现率阶梯器」与预注册证伪方案。书中以标注框区分已发表的证据、可运行的 Python 计算(23 段程序)、原典与作者的假说。

数据口径:书中转化率、访客数与规则条数均为示意参数;全部计算由书中程序生成;本书属于非个人化经营建议。 本书用精确概率推翻这条常识:四十条全部无效的规则同时检验,平均每周冒出两条假冠军,至少一条的概率为 0.8715;模拟中每周最佳假冠军宣称的相对提升平均为 34.2 %,而真实提升为零。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0023, 0.0896

A = 2.7726, B = -1.5581, 24 rules, 4 real
layerfalse/runP(any false)FDRpower
SPRT KEEP0.4240.35400.08960.886
e-BH0.0090.00950.00230.792
Ville+BH0.0540.05150.01200.833
查看完整验算程序
# Chapter 8. p81: a weekly radar that runs one SPRT per copy rule, and
# the multiple-rule layer added on top. Model (illustrative): 24 rules,
# baseline p0 = 0.030 known, mde 20 % (p1 = 0.036), alpha 0.05, beta
# 0.20, Wald bounds A = ln(16), B = ln(0.2/0.95); 1800 visitors per
# rule per week, 8 weeks; four rules truly convert at 0.036, the rest
# at 0.030. Counts use the normal approximation to the binomial.
# Layer 1 (as is): a rule is KEEP once its log-likelihood ratio
# crosses A. Layer 2: at the week-8 review the likelihood ratio of
# each rule is an e-value; e-BH (Wang and Ramdas 2022) rejects the k
# largest with e_(k) >= m / (q k), valid under any dependence.
# Layer 3: Ville p-value 1 / max_t LR_t, then BH. 2000 runs, seed
# 20261113.
import math
import random

M, REAL, P0, MDE, ALPHA, BETA = 24, 4, 0.030, 0.20, 0.05, 0.20
NW, WEEKS, Q, RUNS, SEED = 1800, 8, 0.05, 2000, 20261113
P1 = P0 * (1 + MDE)
A = math.log((1 - BETA) / ALPHA)
B = math.log(BETA / (1 - ALPHA))
UP, DN = math.log(P1 / P0), math.log((1 - P1) / (1 - P0))


def rule_path(rng, p):
    terms, top, keep, decided = [], 0.0, False, False
    for _ in range(WEEKS):
        k = round(NW * p + math.sqrt(NW * p * (1 - p)) * rng.gauss(0, 1))
        k = max(0, k)
        terms.append(k * UP + (NW - k) * DN)
        llr = math.fsum(terms)
        top = max(top, llr)
        if not decided and llr >= A:
            keep, decided = True, True
        elif not decided and llr <= B:
            decided = True
    return keep, math.exp(llr), min(1.0, math.exp(-top))


def ebh(es, q):
    m = len(es)
    idx = sorted(range(m), key=lambda i: (-es[i], i))
    kmax = 0
    for k, i in enumerate(idx, 1):
        if es[i] >= m / (q * k):
            kmax = k
    return set(idx[:kmax])


def bh(ps, q):
    m = len(ps)
    idx = sorted(range(m), key=lambda i: (ps[i], i))
    kmax = 0
    for k, i in enumerate(idx, 1):
        if ps[i] <= k * q / m:
            kmax = k
    return set(idx[:kmax])


if __name__ == "__main__":
    rng = random.Random(20261113)
    stats = {n: ([], [], []) for n in ("SPRT KEEP", "e-BH", "Ville+BH")}
    for _ in range(RUNS):
        res = [rule_path(rng, P1 if i < REAL else P0)
               for i in range(M)]
        sets = {"SPRT KEEP": {i for i in range(M) if res[i][0]},
                "e-BH": ebh([r[1] for r in res], Q),
                "Ville+BH": bh([r[2] for r in res], Q)}
        for n, s in sets.items():
            v = len([i for i in s if i >= REAL])
            stats[n][0].append(float(v))
            stats[n][1].append(v / max(len(s), 1))
            stats[n][2].append((len(s) - v) / REAL)
    print("A = {:.4f}, B = {:.4f}, {} rules, {} real".format(A, B, M, REAL))
    print("layer       false/run  P(any false)  FDR     power")
    for n, (fv, fd, pw) in stats.items():
        anyf = len([x for x in fv if x > 0]) / RUNS
        print("{:10s}  {:.3f}      {:.4f}        {:.4f}  {:.3f}".format(
            n, math.fsum(fv) / RUNS, anyf, math.fsum(fd) / RUNS,
            math.fsum(pw) / RUNS))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《多重比较与伪发现率控制、同时检验数十条文案规则的本杰明尼霍赫伯格阶梯母典》第 8 章「实证闭环:周度复盘雷达的多规则校正与证伪方案」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-307 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026307,
  author    = {Fan, Yafu},
  title     = {Multiple Comparisons and False-Discovery-Rate Control: the Benjamini-Hochberg Step-Up for Testing Dozens of Copy Rules at Once},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论