关于「实证闭环:三十站停止规则仪表、最小样本计算器与证伪方案」,《A/B 测试的偷看与小流量》给出的定量结论是什么?
结论:第八章给出三十站停止规则仪表、A/B 停止规则器的用法与预注册证伪方案。书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。
数据口径:书中的站点、流量、转化率与提升幅度均为示意参数;全部计算由书中程序生成;本书不是个人化的经营、法律或理财建议。 书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 19
| site | vis/d base mde28 d@10% rule | same, next column | |||||
| S01 | 93 2.5% | 80.4% 1,368 NOAB | S16 | 1,094 3.3% | 17.9% | 87 SEQ | |
| S02 | 103 4.5% | 54.0% | 682 NOAB | S17 | 1,197 1.9% | 23.3% | 144 SEQ |
| S03 | 121 4.3% | 50.5% | 606 NOAB | S18 | 1,385 5.5% | 12.1% | 41 SEQ |
| S04 | 141 4.4% | 45.6% | 503 NOAB | S19 | 1,524 2.0% | 19.7% | 104 SEQ |
| S05 | 164 4.8% | 40.1% | 398 NOAB | S20 | 2,252 4.0% | 11.2% | 35 SEQ |
| S06 | 194 5.0% | 35.9% | 324 NOAB | S21 | 2,592 4.9% | 9.3% | 25 FIX |
| S07 | 296 3.5% | 34.8% | 305 NOAB | S22 | 3,832 2.6% | 10.7% | 32 SEQ |
| S08 | 333 2.1% | 43.6% | 462 NOAB | S23 | 4,540 3.8% | 8.1% | 19 FIX |
| S09 | 337 3.2% | 34.5% | 300 NOAB | S24 | 4,767 1.9% | 11.5% | 37 SEQ |
| S10 345 4.4% 28.3% 208 SEQ | S25 8,395 4.1% 5.6% 10 FIX | ||||||
| S11 | 377 4.0% | 28.4% | 209 SEQ | | S26 15,636 3.1% | 4.8% | 7 FIX |
| S12 | 502 1.3% | 46.1% | 510 NOAB | S27 16,934 4.4% | 3.8% | 5 FIX | |
| S13 | 648 3.1% | 24.7% | 161 SEQ | | S28 21,122 1.6% | 5.7% | 10 FIX |
| S14 | 667 5.2% | 18.2% | 90 SEQ | | S29 22,825 1.6% | 5.5% | 9 FIX |
| S15 | 861 4.7% | 16.9% | 78 SEQ | | S30 27,642 1.1% | 6.2% | 11 FIX |
| FIX 8 | SEQ 12 | NOAB 10 | ||||
查看完整验算程序
# Chapter 8. p81: a stopping-rule dashboard for thirty sites
# Model: thirty synthetic sites (seeded) with daily visitors log-uniform
# between 80 and 40,000 and base conversion uniform in 0.01 to 0.06.
# For each: relative MDE in 28 days (p22), days a fixed test needs for a
# 10% lift (p21), and the rule the book recommends:
# FIX : 10% lift is visible within 28 days -> fixed horizon, at most
# 5 O'Brien-Fleming looks (p51)
# SEQ : a 20% lift is visible within 56 days -> test only bold
# changes, with an SPRT or mSPRT (p41, p61)
# NOAB : neither -> no A/B test; pool traffic or change the metric
# All sites and thresholds are illustrative.
import math
import random
from statistics import NormalDist
N01 = NormalDist()
def n_per_arm(p1, rel, alpha=0.05, power=0.80):
p2 = p1 * (1 + rel)
za = N01.inv_cdf(1 - alpha / 2)
zb = N01.inv_cdf(power)
pb = (p1 + p2) / 2
num = za * math.sqrt(2 * pb * (1 - pb)) + zb * math.sqrt(
p1 * (1 - p1) + p2 * (1 - p2))
return (num / (p2 - p1)) ** 2
def mde(p1, n, lo=1e-4, hi=20.0):
for _ in range(80):
mid = (lo + hi) / 2
if n_per_arm(p1, mid) > n:
lo = mid
else:
hi = mid
return hi
def rule(t, p):
if 2 * n_per_arm(p, 0.10) / t <= 28:
return "FIX"
if 2 * n_per_arm(p, 0.20) / t <= 56:
return "SEQ"
return "NOAB"
def main():
rng = random.Random(20261112)
sites = []
for i in range(30):
t = int(math.exp(rng.uniform(math.log(80), math.log(40000))))
p = rng.uniform(0.01, 0.06)
sites.append((t, p))
sites.sort()
rows = []
for i, (t, p) in enumerate(sites, 1):
d10 = math.ceil(2 * n_per_arm(p, 0.10) / t)
m28 = mde(p, t / 2 * 28)
rows.append("S{:02d} {:6,d} {:4.1%} {:6.1%} {:5,d} {:4}".format(
i, t, p, m28, d10, rule(t, p)))
print("site vis/d base mde28 d@10% rule | same, next column")
for a, b in zip(rows[:15], rows[15:]):
print((a + " | " + b).rstrip())
tally = {}
for t, p in sites:
r = rule(t, p)
tally[r] = tally.get(r, 0) + 1
print("FIX {} SEQ {} NOAB {}".format(
tally.get("FIX", 0), tally.get("SEQ", 0), tally.get("NOAB", 0)))
main()唯一数据源与引用
范雅福(Fan, Y.). (2026). 《A/B测试偷看早停假阳性膨胀、序贯概率比检验与小流量站点最小样本母典》第 8 章「实证闭环:三十站停止规则仪表、最小样本计算器与证伪方案」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-217 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026217,
author = {Fan, Yafu},
title = {Peeking-Inflated False Positives in A/B Tests, Sequential Probability Ratio Tests and Minimum Samples for Low-Traffic Sites},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}