关于「偷看的价格:每天一看、显著即停的假阳性精确账」,《A/B 测试的偷看与小流量》给出的定量结论是什么?
结论:第一章用格点递推精确计算:每天看一次、显著即停,两周后的假阳性率是 0.2197,一个月后是 0.2802,并修正了立项时「两到三成」的说法。
数据口径:书中的站点、流量、转化率与提升幅度均为示意参数;全部计算由书中程序生成;本书不是个人化的经营、法律或理财建议。 书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.2197, 0.2802
| A/A test, two-sided z at every look, nominal alpha 0.05 | ||
| looks | P(at least one 'significant') | x nominal |
| 1 | 0.0500 | 1.00 |
| 2 | 0.0831 | 1.66 |
| 3 | 0.1073 | 2.15 |
| 5 | 0.1417 | 2.83 |
| 7 | 0.1663 | 3.33 |
| 10 | 0.1934 | 3.87 |
| 14 | 0.2197 | 4.39 |
| 20 | 0.2480 | 4.96 |
| 30 | 0.2802 | 5.60 |
| 60 | 0.3347 | 6.69 |
| daily for 2 weeks : 0.2197 | ||
| daily for 30 days : 0.2802 | ||
| daily for 60 days : 0.3347 |
查看完整验算程序
# Chapter 1. p11: exact false-positive rate of "look daily, stop if p<0.05"
# Model: an A/A test (no true difference); at every look a two-sided z
# test, stop as soon as |z| >= 1.96. Equal information arrives between
# looks, so S_k is a sum of k iid N(0,1) and z_k = S_k / sqrt(k).
# A grid recursion (the numerical-integration idea of Armitage, McPherson
# and Rowe 1969) gives the probability of at least one 'significant'
# look; grid width 0.05 sd, no Monte Carlo.
# Parameters: alpha 0.05 by convention; number of looks illustrative.
import math
from statistics import NormalDist
N01 = NormalDist()
H = 0.05 # grid width in standard deviations
WIDTH = 7.0 # kernel cut at 7 sd
def cross_curve(bounds, h=H):
"""Cumulative P(|S_k| >= b_k*sqrt(k) at some look <= k)."""
cdf = N01.cdf
m = int(WIDTH / h) + 1
kern = [cdf((d + .5) * h) - cdf((d - .5) * h)
for d in range(-m, m + 1)]
mass, n0 = [1.0], 0
out = []
for k, b in enumerate(bounds, 1):
lim = b * math.sqrt(k)
n = int(lim / h + 0.5)
new = []
for i in range(-n, n + 1):
lo = max((i - .5) * h, -lim)
hi = min((i + .5) * h, lim)
if hi <= lo:
new.append(0.0)
continue
full = lo == (i - .5) * h and hi == (i + .5) * h
acc = []
for jj, w in enumerate(mass):
d = i - (jj - n0)
if w == 0.0 or d < -m or d > m:
continue
if full:
acc.append(w * kern[d + m])
else:
u = (jj - n0) * h
acc.append(w * (cdf(hi - u) - cdf(lo - u)))
new.append(math.fsum(acc))
mass, n0 = new, n
out.append(1.0 - math.fsum(mass))
return out
def main():
z = N01.inv_cdf(0.975)
looks = [1, 2, 3, 5, 7, 10, 14, 20, 30, 60]
curve = cross_curve([z] * looks[-1])
print("A/A test, two-sided z at every look, nominal alpha 0.05")
print("looks P(at least one 'significant') x nominal")
for k in looks:
p = curve[k - 1]
print("{:5d} {:29.4f} {:9.2f}".format(k, p, p / 0.05))
print("daily for 2 weeks : {:.4f}".format(curve[13]))
print("daily for 30 days : {:.4f}".format(curve[29]))
print("daily for 60 days : {:.4f}".format(curve[59]))
main()唯一数据源与引用
范雅福(Fan, Y.). (2026). 《A/B测试偷看早停假阳性膨胀、序贯概率比检验与小流量站点最小样本母典》第 1 章「偷看的价格:每天一看、显著即停的假阳性精确账」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-217 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026217,
author = {Fan, Yafu},
title = {Peeking-Inflated False Positives in A/B Tests, Sequential Probability Ratio Tests and Minimum Samples for Low-Traffic Sites},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}