关于「证据:A/B 测试、幸存者偏差与早期预测」,《完播风险、推荐反馈与获客漏斗》给出的定量结论是什么?

结论:第六章显示付费转化的 A/B 测试每组需要 42,691 名观众,偷看式停止使 0.239 的无效测试被宣告显著;只学头部账号会推荐平均收入更低的惊悚钩子(RM 1,250 对 RM 1,647);以前三天预测全年会低估 0.602。

数据口径:书中的威布尔参数、流量池规模、转化率、价格(令吉 RM)与封号概率都是示意模型参数,不是任何平台、账号或企业的数据;本书不描述任何平台的真实算法,也不构成营销或经营建议。 第二章模拟分级流量池:300 人样本下,即使各池观众相同,好视频也有 0.169 在第一关被拦下,观众差异下为 0.308 至 0.478;对冲噪声应重剪而不是重发。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.239, 42691

   outcome            p1      p2     viewers per arm
   5 s retention     0.600   0.630           4,126
   completion        0.200   0.220           6,507
   paid conversion   0.010   0.012          42,691
   peeking daily for 20 days with no true difference:
   share of tests declared 'significant' 0.239
Reading: an opening that lifts early retention by three
points can be tested in a few days of ordinary traffic; a
change in paid conversion needs tens of thousands of viewers
per arm. Checking every day and stopping at the first good
number turns a 0.05 test into roughly a one in four false
alarm.
查看完整验算程序
# Chapter 6. p11: How many impressions does a hook A/B test need?
# Two openings of the same video are shown to random halves of the
# audience. Per arm, for a two-sided test at 0.05 with power 0.80:
#   n = (1.96 + 0.8416)^2 (p1 (1 - p1) + p2 (1 - p2)) / (p2 - p1)^2.
# Outcomes: 5-second retention 0.60 vs 0.63; completion 0.20 vs 0.22;
# paid conversion 0.010 vs 0.012 per viewer. Then the cost of peeking:
# under no true difference (both arms at 0.20), a creator checks the
# running z statistic after each of 20 daily batches of 500 viewers
# per arm and stops at the first |z| > 1.96. 4,000 simulated tests.
import math
import random

Z = 1.96 + 0.8416

def n_arm(p1, p2):
    return math.ceil(Z ** 2 * (p1 * (1 - p1) + p2 * (1 - p2)) /
                     (p2 - p1) ** 2)

if __name__ == "__main__":
    print("   outcome            p1      p2     viewers per arm")
    for name, p1, p2 in (("5 s retention", 0.60, 0.63),
                         ("completion", 0.20, 0.22),
                         ("paid conversion", 0.010, 0.012)):
        print("   {:16s} {:6.3f} {:7.3f} {:15,d}".format(
            name, p1, p2, n_arm(p1, p2)))
    rng = random.Random(20)
    stops = 0
    for _ in range(2000):
        a = b = n = 0
        for day in range(20):
            for _ in range(500):
                a += rng.random() < 0.2
                b += rng.random() < 0.2
            n += 500
            pa, pb = a / n, b / n
            pool = (a + b) / (2 * n)
            se = math.sqrt(pool * (1 - pool) * 2 / n)
            if se > 0 and abs(pa - pb) / se > 1.96:
                stops += 1
                break
    print("\n   peeking daily for 20 days with no true difference:")
    print("   share of tests declared 'significant' {:.3f}".format(
        stops / 2000))
    print("\nReading: an opening that lifts early retention by three")
    print("points can be tested in a few days of ordinary traffic; a")
    print("change in paid conversion needs tens of thousands of viewers")
    print("per arm. Checking every day and stopping at the first good")
    print("number turns a 0.05 test into roughly a one in four false")
    print("alarm.")

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《短视频完播率风险函数、推荐反馈环与自动化获客漏斗》第 6 章「证据:A/B 测试、幸存者偏差与早期预测」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090213

DOI: 10.5281/zenodo.24090213 · 阅读本章所在专著 · 5DL-B-081 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026081,
  author    = {Fan, Yafu},
  title     = {Short-Video Completion Hazards, Recommender Exploration Loops and Automated Acquisition Funnels},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.24090213},
  url       = {https://doi.org/10.5281/zenodo.24090213}
}

本书其他章节结论