What does Creative Fatigue and Orthogonal Remixing conclude, in numbers, about “How Many Winners Can a Hundred Creatives Reveal? Sample Size, the False Discovery Rate and the Signal-to-Noise Ratio of Advertising Returns”?
Answer: Chapter 5 computes sample sizes and the false discovery rate: detecting a ten per cent lift needs 163,095 impressions per creative.
Scope of the numbers: Parameters are illustrative; all calculations come from the book's programs. This is general business analysis, not individual advice, and promises no advertising result.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 163095
| relative lift | CTR a=0.05 | CTR a=0.0005 | buy a=0.05 |
| 0.05 | 637,010 | 1,516,297 | 12,865,565 |
| 0.10 | 163,095 | 388,221 | 3,294,798 |
| 0.20 | 42,693 | 101,625 | 862,901 |
| 0.30 | 19,827 | 47,195 | 400,934 |
| 100 creatives + control at lift 0.10, CTR: 16,472,595 impressions | |||
| same on purchases with Bonferroni: 792,114,417 impressions |
Show the full program
# Chapter 5. p51: impressions per creative needed to see a CTR lift
# Two-sided two-proportion z test, power 0.80. Baselines are
# illustrative: CTR 0.010 per impression, purchase rate 0.0005 per
# impression. Per-test alpha 0.05, and 0.05 / 100 when a hundred
# creatives are each compared with the control (Bonferroni bound).
import math
import statistics
Z = statistics.NormalDist()
def n_per_arm(p1, lift, alpha, power=0.80):
p2 = p1 * (1.0 + lift)
pb = (p1 + p2) / 2.0
za = Z.inv_cdf(1.0 - alpha / 2.0)
zb = Z.inv_cdf(power)
num = za * math.sqrt(2 * pb * (1 - pb)) + \
zb * math.sqrt(p1 * (1 - p1) + p2 * (1 - p2))
return math.ceil(num ** 2 / (p2 - p1) ** 2)
print("relative lift CTR a=0.05 CTR a=0.0005 buy a=0.05")
for lift in (0.05, 0.10, 0.20, 0.30):
print("{:>13.2f} {:>11,} {:>13,} {:>11,}".format(
lift, n_per_arm(0.010, lift, 0.05), n_per_arm(0.010, lift, 0.0005),
n_per_arm(0.0005, lift, 0.05)))
n10 = n_per_arm(0.010, 0.10, 0.05)
print("100 creatives + control at lift 0.10, CTR: {:,} impressions".format(
101 * n10))
nb = n_per_arm(0.0005, 0.10, 0.0005)
print("same on purchases with Bonferroni: {:,} impressions".format(
101 * nb))Single source and citation
Fan, Y. (2026). Ad-Creative Fatigue Half-Lives, Orthogonal Storyboard Recombination and Automated Permutation Rendering (Version 1.0.0), Chapter 5: How Many Winners Can a Hundred Creatives Reveal? Sample Size, the False Discovery Rate and the Signal-to-Noise Ratio of Advertising Returns. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-400 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026400,
author = {Fan, Yafu},
title = {Ad-Creative Fatigue Half-Lives, Orthogonal Storyboard Recombination and Automated Permutation Rendering},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Fatigue Curve Is Not an Exponential: the Two-Factor Process of Repetition and the Estimation of Half-Lives
- Chapter 2 · Pausing Brings Recovery: Naik-Type Dynamics, Frequency Caps and the Size of a Rotation
- Chapter 3 · What an Orthogonal Array Can Estimate: Building L4, L8, L9 and L16, Main-Effect Efficiency and the Structure of Aliasing
- Chapter 4 · How Different Are Recombined Creatives? Hamming Distance, Correlated Performance and the Shared Fatigue of Shared Shots
- Chapter 6 · The Platform Will Not Run Your Experiment: the Bias of Allocating by Prediction and Thompson-Sampling Races
- Chapter 7 · Classical Sources: the Changes of Five Notes, the Art of Combination and Nothing New Under the Sun
- Chapter 8 · Closing the Loop: the Fatigue-and-Remix Ledger, Field Tools and Checker Specifications