What does Case-Screen Hooks and Paywall Funnels conclude, in numbers, about “Hook Tournaments: Batched Bandits, Multiple Comparisons and Honest Stopping”?

Answer: Chapter 4 runs hooks as a batched bandit tournament; Thompson sampling cuts the plays lost from 6,883 to 952; with twelve candidates the naive rule produces at least one false winner with probability 0.252, false-discovery-rate control brings the share of false winners among winners to 0.066, peeking pushes the false-positive rate to 0.270 and a sequential test brings it back to 0.010; the minimum sample is 2,311 plays per arm.

Scope of the numbers: In a synthetic year of each strategy, steady case videos matched to their audience bring 16.51 times as many payers a day as betting on viral hits. It is a target, not a promise: at an illustrative baseline it needs 280,112 plays a day.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.066, 0.252, 2311

A. 1,000 tournaments12 hooks, 3 real lifts (0.60 -> 0.64), n = 2,000 each
rule false wins true wins false-discovery share any false win
p < 0.050.423 2.271 0.122 0.252
BH q 0.100.264 1.807 0.066 0.132
B. Single-test power for a 0.04 lift at n2,000: 0.741
plays per arm for power 0.82,311
Show the full program
# Chapter 4. p42: twelve hooks against a control, with and without false-discovery control.
# Each hook and the control get n = 2,000 plays; the outcome is staying past second 3.
# Control rate 0.60. Three hooks carry a real lift to 0.64; nine are exactly null.
# Two-sided two-proportion z-tests. Compare "every p < 0.05 is a winner" with the
# Benjamini-Hochberg procedure at q = 0.10. 1,000 synthetic tournaments, fixed seed.
import math
import random

N, P0 = 2000, 0.60
TRUE = [0.64, 0.64, 0.64] + [0.60] * 9


def norm_sf(z):
    return 0.5 * math.erfc(z / math.sqrt(2.0))


def count(rng, n, p):
    x = int(round(rng.gauss(n * p, math.sqrt(n * p * (1 - p)))))
    return min(n, max(0, x))


def pval(k1, k0, n):
    p1, p0 = k1 / n, k0 / n
    pp = (k1 + k0) / (2.0 * n)
    se = math.sqrt(pp * (1 - pp) * 2.0 / n)
    return 2.0 * norm_sf(abs(p1 - p0) / se)


def bh(ps, q):
    m = len(ps)
    order = sorted(range(m), key=lambda i: ps[i])
    kmax = 0
    for rank, i in enumerate(order, start=1):
        if ps[i] <= q * rank / m:
            kmax = rank
    return set(order[:kmax])


if __name__ == "__main__":
    rng = random.Random(20261120)
    stats = {"naive": [], "bh": []}
    for _ in range(1000):
        k0 = count(rng, N, P0)
        ps = [pval(count(rng, N, p), k0, N) for p in TRUE]
        naive = {i for i, p in enumerate(ps) if p < 0.05}
        for name, rej in (("naive", naive), ("bh", bh(ps, 0.10))):
            false = len([i for i in rej if TRUE[i] == P0])
            true = len(rej) - false
            stats[name].append((false, true, false / len(rej) if rej else 0.0, false > 0))
    print("A. 1,000 tournaments: 12 hooks, 3 real lifts (0.60 -> 0.64), n = 2,000 each")
    print("  rule        false wins   true wins   false-discovery share   any false win")
    for name in ("naive", "bh"):
        s = stats[name]
        f = math.fsum([x[0] for x in s]) / len(s)
        t = math.fsum([x[1] for x in s]) / len(s)
        fdr = math.fsum([x[2] for x in s]) / len(s)
        fw = len([1 for x in s if x[3]]) / len(s)
        print("  {:<10} {:>10.3f} {:>11.3f} {:>18.3f} {:>16.3f}".format(
            "p < 0.05" if name == "naive" else "BH q 0.10", f, t, fdr, fw))
    print()
    z = (0.64 - 0.60) / math.sqrt(0.62 * 0.38 * 2.0 / N)
    power = norm_sf(1.959964 - z) + norm_sf(1.959964 + z)
    print("B. Single-test power for a 0.04 lift at n = 2,000: {:.3f}".format(power))
    za, zb = 1.959964, 0.841621
    n80 = 2.0 * (za + zb) ** 2 * 0.62 * 0.38 / 0.04 ** 2
    print("  plays per arm for power 0.8: {:,.0f}".format(n80))

Single source and citation

Fan, Y. (2026). Faceless Case-Screen Short-Video Production, Three-Second Hook Tournaments and Direct-to-Paywall Traffic Funnels (Version 1.0.0), Chapter 4: Hook Tournaments: Batched Bandits, Multiple Comparisons and Honest Stopping. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-350 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026350,
  author    = {Fan, Yafu},
  title     = {Faceless Case-Screen Short-Video Production, Three-Second Hook Tournaments and Direct-to-Paywall Traffic Funnels},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book