What does Creative Fatigue and Orthogonal Remixing conclude, in numbers, about “How Many Winners Can a Hundred Creatives Reveal? Sample Size, the False Discovery Rate and the Signal-to-Noise Ratio of Advertising Returns”?

Answer: Chapter 5 computes sample sizes and the false discovery rate: detecting a ten per cent lift needs 163,095 impressions per creative.

Scope of the numbers: Parameters are illustrative; all calculations come from the book's programs. This is general business analysis, not individual advice, and promises no advertising result.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 163095

relative liftCTR a=0.05CTR a=0.0005buy a=0.05
0.05637,0101,516,29712,865,565
0.10163,095388,2213,294,798
0.2042,693101,625862,901
0.3019,82747,195400,934
100 creatives + control at lift 0.10, CTR: 16,472,595 impressions
same on purchases with Bonferroni: 792,114,417 impressions
Show the full program
# Chapter 5. p51: impressions per creative needed to see a CTR lift
# Two-sided two-proportion z test, power 0.80. Baselines are
# illustrative: CTR 0.010 per impression, purchase rate 0.0005 per
# impression. Per-test alpha 0.05, and 0.05 / 100 when a hundred
# creatives are each compared with the control (Bonferroni bound).
import math
import statistics

Z = statistics.NormalDist()


def n_per_arm(p1, lift, alpha, power=0.80):
    p2 = p1 * (1.0 + lift)
    pb = (p1 + p2) / 2.0
    za = Z.inv_cdf(1.0 - alpha / 2.0)
    zb = Z.inv_cdf(power)
    num = za * math.sqrt(2 * pb * (1 - pb)) + \
        zb * math.sqrt(p1 * (1 - p1) + p2 * (1 - p2))
    return math.ceil(num ** 2 / (p2 - p1) ** 2)


print("relative lift  CTR a=0.05  CTR a=0.0005  buy a=0.05")
for lift in (0.05, 0.10, 0.20, 0.30):
    print("{:>13.2f} {:>11,} {:>13,} {:>11,}".format(
        lift, n_per_arm(0.010, lift, 0.05), n_per_arm(0.010, lift, 0.0005),
        n_per_arm(0.0005, lift, 0.05)))
n10 = n_per_arm(0.010, 0.10, 0.05)
print("100 creatives + control at lift 0.10, CTR: {:,} impressions".format(
    101 * n10))
nb = n_per_arm(0.0005, 0.10, 0.0005)
print("same on purchases with Bonferroni: {:,} impressions".format(
    101 * nb))

Single source and citation

Fan, Y. (2026). Ad-Creative Fatigue Half-Lives, Orthogonal Storyboard Recombination and Automated Permutation Rendering (Version 1.0.0), Chapter 5: How Many Winners Can a Hundred Creatives Reveal? Sample Size, the False Discovery Rate and the Signal-to-Noise Ratio of Advertising Returns. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-400 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026400,
  author    = {Fan, Yafu},
  title     = {Ad-Creative Fatigue Half-Lives, Orthogonal Storyboard Recombination and Automated Permutation Rendering},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book