5D Library › Short-Video Completion Hazards, Recommender Exploration Loops and Automated Acquisition Funnels · 中文
Chapter 8. A Ten-Flag Audit and a Pre-Registered Experiment Randomised by Video
Ten questions
This chapter turns the book into ten questions a script can check against any business that acquires customers through short video, the author’s own included. They are not marketing or business advice.
- Are mid-video re-hooks closer than one every 10 seconds? (Chapter 1)
- Are duplicates reposted unchanged where a fifth or more of them are caught? (Chapter 2)
- Are videos ranked by raw completion or raw watch time? (Chapter 3)
- Is hook breadth above 0.5? (Chapter 4)
- Do viewers see the offer the moment they arrive after a cliffhanger? (Chapter 4)
- Do the accounts share devices or payment details? (Chapter 5)
- Is the cash reserve below 48 days of costs? (Chapter 5)
- Are A/B tests stopped on the first good-looking day? (Chapter 6)
- Are tactics copied only from top accounts? (Chapter 6)
- Is the year planned from the first three days of views? (Chapter 6)
The audit as a program
The program writes the ten questions as ten automatic flags and applies them to two illustrative operations: a “naive” one (common practice) and an “audited” one corrected chapter by chapter in this book.
Calculation
# Chapter 8. p15: Ten automatic flags for a short-video sales funnel.
# Each flag is a yes / no check on an operation's own settings; the
# chapter of this book that supplies the threshold is in brackets.
# Two illustrative operations: "naive" (common practice) and
# "audited" (after the corrections of this book).
FLAGS = [
("re-hooks closer than every 10 s [1]",
lambda o: o["hook_gap_s"] < 10),
("duplicates reposted where 20 % or more are caught [2]",
lambda o: o["reposts"] and o["dup_catch"] >= 0.2),
("videos ranked by raw completion or raw watch time [3]",
lambda o: o["metric"] in ("completion", "watch")),
("hook breadth above 0.5 [4]",
lambda o: o["breadth"] > 0.5),
("offer shown at arrival after an open story [4]",
lambda o: o["offer_delay_s"] < 2 and o["open_story"]),
("accounts share devices or payment details [5]",
lambda o: o["linked"]),
("cash reserve under 48 days of costs [5]",
lambda o: o["reserve_days"] < 48),
("A/B tests stopped at the first good day [6]",
lambda o: o["peeking"]),
("tactics copied from top accounts only [6]",
lambda o: o["copy_top"]),
("year planned from the first three days of views [6]",
lambda o: o["forecast_days"] <= 3),
]
OPS = {
"naive": dict(hook_gap_s=4, reposts=True, dup_catch=0.5,
metric="completion", breadth=1.0, offer_delay_s=0,
open_story=True, linked=True, reserve_days=14,
peeking=True, copy_top=True, forecast_days=3),
"audited": dict(hook_gap_s=10, reposts=False, dup_catch=0.5,
metric="normalised", breadth=0.314, offer_delay_s=8,
open_story=True, linked=False, reserve_days=60,
peeking=False, copy_top=False, forecast_days=60),
}
if __name__ == "__main__":
for name, op in OPS.items():
hits = [f for f, test in FLAGS if test(op)]
print(" {}: {} of {} flags".format(name, len(hits), len(FLAGS)))
for f in hits:
print(" [x] " + f)
print("\nReading: the naive operation trips every flag. The audited")
print("one trips none, which says only that its settings agree with")
print("this book's models; whether its buyers are better served is")
print("a question for the experiment of p16.")
# naive: 10 of 10 flags
# [x] re-hooks closer than every 10 s [1]
# [x] duplicates reposted where 20 % or more are caught [2]
# [x] videos ranked by raw completion or raw watch time [3]
# [x] hook breadth above 0.5 [4]
# [x] offer shown at arrival after an open story [4]
# [x] accounts share devices or payment details [5]
# [x] cash reserve under 48 days of costs [5]
# [x] A/B tests stopped at the first good day [6]
# [x] tactics copied from top accounts only [6]
# [x] year planned from the first three days of views [6]
# audited: 0 of 10 flags
#
# Reading: the naive operation trips every flag. The audited
# one trips none, which says only that its settings agree with
# this book's models; whether its buyers are better served is
# a question for the experiment of p16.The naive operation trips all ten flags. The audited one trips none, but that says only that its settings agree with this book’s models, not that its buyers are better served. Only an experiment can answer that.
A pre-registered experiment randomised by video
Hypothesis · not established
Experiment protocol. Each new video is randomly given a narrow hook (breadth 0.3) or a broad one (breadth 1.0). Viewers of one video share that video’s audience draw, so the unit of analysis is the video, not the viewer. Primary outcome: paid conversion per viewer, assumed 0.010 under the narrow hook and 0.008 under the broad one; two-sided 0.05, power 0.80; a video-level t test. Carried out with pre-registered code by an independent analyst with no financial relationship with the author; results published whether positive or negative.
Calculation
# Chapter 8. p16: A pre-registered, video-randomised hook experiment.
# Each new video is randomly given a narrow hook (breadth 0.3) or a
# broad one (breadth 1.0). Viewers of one video share its audience
# draw, so the unit of analysis is the video, not the viewer.
# Primary outcome: paid conversion per viewer, assumed 0.010 under the
# narrow hook and 0.008 under the broad one at equal reach. Two-sided
# 0.05, power 0.80. Viewers per arm ignoring clustering:
# n = (1.96 + 0.8416)^2 (p1 (1 - p1) + p2 (1 - p2)) / (p1 - p2)^2;
# with m viewers per video and intraclass correlation icc, multiply by
# the design effect 1 + (m - 1) icc and divide by m to get videos.
# A simulation of 2,000 null experiments (both arms at 0.010, icc
# 0.002, m = 2,000, 60 videos per arm) checks the type I error of a
# video-level t test.
import math
import random
Z = 1.96 + 0.8416
def viewers(p1, p2):
return Z ** 2 * (p1 * (1 - p1) + p2 * (1 - p2)) / (p1 - p2) ** 2
if __name__ == "__main__":
n = viewers(0.010, 0.008)
print(" viewers per arm if viewers were independent: {:,.0f}".format(
math.ceil(n)))
print(" m viewers/video icc design effect videos per arm")
for m in (500, 2000, 8000):
for icc in (0.0005, 0.002):
de = 1 + (m - 1) * icc
print(" {:15,d} {:7.4f} {:15.2f} {:16d}".format(
m, icc, de, math.ceil(n * de / m)))
rng = random.Random(16)
sd_p = math.sqrt(0.002 * 0.010 * 0.990)
rej = 0
for _ in range(2000):
arms = []
for _ in range(2):
rates = []
for _ in range(60):
p = min(max(rng.gauss(0.010, sd_p), 0.0), 1.0)
k = rng.gauss(2000 * p, math.sqrt(2000 * p * (1 - p)))
rates.append(k / 2000)
arms.append(rates)
ma, mb = (sum(a) / 60 for a in arms)
va = sum((x - ma) ** 2 for x in arms[0]) / 59
vb = sum((x - mb) ** 2 for x in arms[1]) / 59
t = (ma - mb) / math.sqrt(va / 60 + vb / 60)
rej += abs(t) > 1.98
print(" null simulation, video-level t test: false alarms {:.4f}"
.format(rej / 2000))
print("\nReading: the experiment must count videos, not viewers:")
print("between 22 and 140 videos per arm here, depending on how")
print("much one video's audience differs from another's. Analysed")
print("at the video level, the test keeps its stated error rate.")
# viewers per arm if viewers were independent: 34,999
# m viewers/video icc design effect videos per arm
# 500 0.0005 1.25 88
# 500 0.0020 2.00 140
# 2,000 0.0005 2.00 35
# 2,000 0.0020 5.00 88
# 8,000 0.0005 5.00 22
# 8,000 0.0020 17.00 75
# null simulation, video-level t test: false alarms 0.0465
#
# Reading: the experiment must count videos, not viewers:
# between 22 and 140 videos per arm here, depending on how
# much one video's audience differs from another's. Analysed
# at the video level, the test keeps its stated error rate.If viewers were independent, each arm would need 34,999 viewers. But viewers of the same video are not independent: with 2,000 viewers per video and an intraclass correlation of 0.002 the design effect is 5.00 and each arm needs 88 videos; with an intraclass correlation of 0.0005 it needs 35. With 8,000 viewers per video the two correlations need 75 and 22 videos. In 2,000 simulated null experiments the video-level t test raises a false alarm in 0.0465 of them, in line with its nominal level.
Summary
Ten questions (re-hook density, reposts, metrics, hook breadth, timing of the offer, linked accounts, cash reserve, peeking, survivorship and early forecasts) can all be written as an automatic audit. An experiment on hook breadth must be randomised and analysed by video; what it needs is dozens to more than a hundred videos, not just a count of viewers.
Fan, Y. (2026). Short-Video Completion Hazards, Recommender Exploration Loops and Automated Acquisition Funnels: Weibull Swipe-Away Kinetics, Multi-Stage Pool Promotion Thresholds and Traffic-to-Cashflow Conversion Bounds [短视频完播率风险函数、推荐反馈环与自动化获客漏斗] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090213 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).