What does Creative Fatigue and Orthogonal Remixing conclude, in numbers, about “The Platform Will Not Run Your Experiment: the Bias of Allocating by Prediction and Thompson-Sampling Races”?
Answer: Chapter 6 shows that under adaptive platform allocation the reported relative difference between two identical creatives misses the truth by 0.348 on average, and that Thompson sampling lowers regret from 317.8 clicks under an equal split to a median of 187.1, at the price of biased estimates of the differences.
Scope of the numbers: Parameters are illustrative; all calculations come from the book's programs. This is general business analysis, not individual advice, and promises no advertising result.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 187.1, 317.8
impressions per campaign 200,000 equal split regret 317.8 clicks Thompson regret 5/50/95%: 118.8 187.1 301.3 clicks Thompson: best creative most shown in 0.793 of campaigns best creative impression share 5/50/95%: 0.070 0.435 0.665
Show the full program
# Chapter 6. p62: Thompson sampling versus an equal split, nine creatives
# Nine L9 creatives; true CTRs are illustrative (one at 0.0120, the rest
# between 0.0095 and 0.0110). 200 rounds of 1,000 impressions. Batched
# Thompson sampling (Thompson 1933): each round draw one Beta posterior
# sample per creative and give the whole batch to the largest draw.
# Regret = clicks lost against always showing the best creative.
import math
import random
rng = random.Random(40062)
TRUE = [0.0120, 0.0110, 0.0108, 0.0105, 0.0103, 0.0100, 0.0099, 0.0097,
0.0095]
ROUNDS, BATCH, RUNS = 200, 1000, 300
BEST = max(TRUE)
def clicks(n, p):
return max(0, round(rng.gauss(n * p, math.sqrt(n * p * (1 - p)))))
def thompson():
a = [1] * 9
b = [1] * 9
regret = []
for _ in range(ROUNDS):
draws = [rng.betavariate(a[k], b[k]) for k in range(9)]
k = draws.index(max(draws))
c = clicks(BATCH, TRUE[k])
a[k] += c
b[k] += BATCH - c
regret.append((BEST - TRUE[k]) * BATCH)
pulls = [a[k] + b[k] - 2 for k in range(9)]
return math.fsum(regret), pulls.index(max(pulls)) == 0, pulls[0]
def equal_split():
per = ROUNDS * BATCH / 9
return math.fsum((BEST - p) * per for p in TRUE)
res = [thompson() for _ in range(RUNS)]
reg = sorted(r[0] for r in res)
share = sorted(r[2] / (ROUNDS * BATCH) for r in res)
print("impressions per campaign {:,}".format(ROUNDS * BATCH))
print("equal split regret {:.1f} clicks".format(equal_split()))
print("Thompson regret 5/50/95%: {:.1f} {:.1f} {:.1f} clicks".format(
reg[RUNS // 20], reg[RUNS // 2], reg[RUNS * 19 // 20]))
print("Thompson: best creative most shown in {:.3f} of campaigns".format(
len([1 for r in res if r[1]]) / RUNS))
print("best creative impression share 5/50/95%: {:.3f} {:.3f} {:.3f}".format(
share[RUNS // 20], share[RUNS // 2], share[RUNS * 19 // 20]))Single source and citation
Fan, Y. (2026). Ad-Creative Fatigue Half-Lives, Orthogonal Storyboard Recombination and Automated Permutation Rendering (Version 1.0.0), Chapter 6: The Platform Will Not Run Your Experiment: the Bias of Allocating by Prediction and Thompson-Sampling Races. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-400 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026400,
author = {Fan, Yafu},
title = {Ad-Creative Fatigue Half-Lives, Orthogonal Storyboard Recombination and Automated Permutation Rendering},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Fatigue Curve Is Not an Exponential: the Two-Factor Process of Repetition and the Estimation of Half-Lives
- Chapter 2 · Pausing Brings Recovery: Naik-Type Dynamics, Frequency Caps and the Size of a Rotation
- Chapter 3 · What an Orthogonal Array Can Estimate: Building L4, L8, L9 and L16, Main-Effect Efficiency and the Structure of Aliasing
- Chapter 4 · How Different Are Recombined Creatives? Hamming Distance, Correlated Performance and the Shared Fatigue of Shared Shots
- Chapter 5 · How Many Winners Can a Hundred Creatives Reveal? Sample Size, the False Discovery Rate and the Signal-to-Noise Ratio of Advertising Returns
- Chapter 7 · Classical Sources: the Changes of Five Notes, the Art of Combination and Nothing New Under the Sun
- Chapter 8 · Closing the Loop: the Fatigue-and-Remix Ledger, Field Tools and Checker Specifications