What does Attribution, Bandits and Customer Lifetime Value conclude, in numbers, about “Practice and a Pre-Registered Test”?
Answer: Chapter 8 gives a four-step checklist (attribution, experiments, lifetime value, budget) and a pre-registered simulation benchmark: on a synthetic multichannel data generator with known true increments, frozen by an independent third party, observational Shapley attribution is compared with last click on their error against the true incremental shares (single primary outcome the difference in mean absolute errors, smallest effect of interest 2 percentage points, 79 synthetic worlds for 90 % power, 90 planned, decision rule written as code). Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's models and hypotheses; all data in the book are fictional or simulated.
Scope of the numbers: Which advertising channel should a sale be credited to? When may an A/B test stop? What is a customer still worth? This book presents mathematical models for teaching and research and is not marketing, investment or legal advice. Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's models and hypotheses; all data in the book are fictional or simulated.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 79, 90
pilot (120 worlds, 20,000 users each): MAE last click 13.0%, Shapley 4.7%
D = MAE(last) - MAE(Shapley): mean +8.28%, sd 5.48%; Shapley better in 112/120
secondary, truth = normalised removal increments v*(N) - v*(N - c):
mean D +8.02%, Shapley better in 112/120
SESOI 2 percentage points; 90 % power, two-sided alpha 0.05:
79 worlds (normal approximation, sd from the pilot)
simulated power at n = 79: 0.899 (4,000 resamples of the pilot differences)
plus 10 % for failed or degenerate worlds, rounded up: plan 90 worlds
sensitivity (worlds) SESOI: 1 pt 2 pt 3 pt
sd as in the pilot 316 79 36
sd 50 % higher 711 178 79Show the full program
# Sample size for the Chapter 8 pre-registered simulation benchmark. Unit: a synthetic
# world drawn from a generator frozen by an independent third party. In each world the
# TRUE credit of a channel is the Shapley value of the interventional game v*(S) =
# expected conversions if exactly the channels in S were switched on (known exactly
# from the generator; for additive worlds it equals each channel's incremental
# conversions). Primary outcome per world: D = MAE(last click) - MAE(observational
# Shapley), the mean absolute error of channel shares against the truth. Secondary
# (sensitivity): the same D against normalised removal increments v*(N) - v*(N - c).
# PILOT GENERATOR (a planning assumption, not the trial's generator): 3-5 channels,
# random targeting strength, lifts, exposure and an optional two-channel synergy.
import itertools, math, random, statistics
# --- helpers repeated from earlier programs, so that this one runs on its own ---
def shapley(players, v):
n, phi = len(players), {}
for i in players:
rest, s = [p for p in players if p != i], 0.0
for k in range(n):
w = math.factorial(k) * math.factorial(n - k - 1) / math.factorial(n)
for T in itertools.combinations(rest, k):
s += w * (v(frozenset(T) | {i}) - v(frozenset(T)))
phi[i] = s
return phi
# --- end of repeated helpers ---
Z = statistics.NormalDist().inv_cdf
def world(rng):
K = rng.choice((3, 4, 5)); ch = "ABCDE"[:K]; ph = rng.uniform(0.1, 0.3)
sel = rng.random()
lo = {c: rng.uniform(0.05, 0.5) for c in ch}
hi = {c: min(0.95, lo[c] + sel * rng.uniform(0, 0.5)) for c in ch}
lift = {c: rng.uniform(0.0, 0.03) for c in ch}
when = {c: rng.random() for c in ch} # typical position in a path
syn = (rng.sample(ch, 2), rng.uniform(0, 0.04)) if rng.random() < 0.5 else (None, 0)
base = (rng.uniform(0.03, 0.10), rng.uniform(0.002, 0.01))
return ch, ph, lo, hi, lift, when, syn, base
def v_star(Wd, S):
ch, ph, lo, hi, lift, when, syn, base = Wd
out = 0.0
for w, e, b in ((ph, hi, base[0]), (1 - ph, lo, base[1])):
x = b + sum(e[c] * lift[c] for c in S)
if syn[0] and all(c in S for c in syn[0]):
x += e[syn[0][0]] * e[syn[0][1]] * syn[1]
out += w * x
return out
def users(rng, Wd, n):
ch, ph, lo, hi, lift, when, syn, base = Wd
data = []
for _ in range(n):
h = rng.random() < ph; e = hi if h else lo
seen = [c for c in ch if rng.random() < e[c]]
path = sorted(seen, key=lambda c: when[c] + 0.3 * rng.random())
p = base[0 if h else 1] + sum(lift[c] for c in seen)
if syn[0] and all(c in seen for c in syn[0]):
p += syn[1]
data.append((path, rng.random() < p))
return data
def mae(a, b, ch):
return sum(abs(a[c] - b[c]) for c in ch) / len(ch)
def score(rng, n_users=20_000):
Wd = world(rng); ch = Wd[0]
tru = shapley(list(ch), lambda S: v_star(Wd, S) - v_star(Wd, ()))
data = users(rng, Wd, n_users)
last = {c: 0.0 for c in ch}
for path, y in data:
if y and path:
last[path[-1]] += 1
cnt, conv = {}, {}
for path, y in data:
S = frozenset(path); cnt[S] = cnt.get(S, 0) + 1; conv[S] = conv.get(S, 0) + y
rate = {S: conv[S] / cnt[S] for S in cnt}
v = lambda T: rate.get(frozenset(T), 0.0) - rate.get(frozenset(), 0.0)
obs = {c: 0.0 for c in ch}
for S, m in cnt.items():
if S:
for c, x in shapley(sorted(S), v).items():
obs[c] += m * x
norm = lambda d: {c: d[c] / sum(d.values()) if sum(d.values()) else 1 / len(ch)
for c in ch}
t, l, s = norm({c: max(tru[c], 0.0) for c in ch}), norm(last), norm(obs)
full = v_star(Wd, ch) # alternative truth: removal increments
alt = norm({c: max(full - v_star(Wd, [d for d in ch if d != c]), 0.0) for c in ch})
return mae(l, t, ch), mae(s, t, ch), mae(l, alt, ch) - mae(s, alt, ch)
if __name__ == "__main__":
rng = random.Random(4800); n0 = 120
R = [score(rng) for _ in range(n0)]
D = [a - b for a, b, _ in R]
m, sd = statistics.fmean(D), statistics.stdev(D)
print(f"pilot ({n0} worlds, 20,000 users each): MAE last click"
f" {statistics.fmean(r[0] for r in R):.1%},"
f" Shapley {statistics.fmean(r[1] for r in R):.1%}")
print(f" D = MAE(last) - MAE(Shapley): mean {m:+.2%}, sd {sd:.2%};"
f" Shapley better in {sum(d > 0 for d in D)}/{n0}")
Da = [r[2] for r in R]
print(f" secondary, truth = normalised removal increments v*(N) - v*(N - c):"
f"\n mean D {statistics.fmean(Da):+.2%}, Shapley better in"
f" {sum(d > 0 for d in Da)}/{n0}")
sesoi = 0.02
n = math.ceil(((Z(0.975) + Z(0.90)) * sd / sesoi) ** 2)
print(f"SESOI 2 percentage points; 90 % power, two-sided alpha 0.05:")
print(f" {n} worlds (normal approximation, sd from the pilot)")
hits, rs = 0, random.Random(9)
for _ in range(4000): # resampling check of power
s = [d - m + sesoi for d in rs.choices(D, k=n)] # pilot shape, mean = SESOI
se = statistics.stdev(s) / math.sqrt(n)
hits += statistics.fmean(s) - Z(0.975) * se > 0
print(f" simulated power at n = {n}: {hits / 4000:.3f} (4,000 resamples of the"
f" pilot differences)")
plan = math.ceil(n * 1.10 / 10) * 10
print(f" plus 10 % for failed or degenerate worlds, rounded up: plan {plan} worlds")
print("sensitivity (worlds) SESOI: 1 pt 2 pt 3 pt")
for lab, f in (("sd as in the pilot", 1.0), ("sd 50 % higher", 1.5)):
row = [math.ceil(((Z(0.975) + Z(0.90)) * sd * f / e) ** 2) for e in
(0.01, 0.02, 0.03)]
print(f" {lab:28s}" + "".join(f"{x:8,d}" for x in row))Single source and citation
Fan, Y. (2026). Algorithmic Marketing, Multi-Touch Shapley Attribution and Stochastic Control of Acquisition Funnels (Version 1.0.0), Chapter 8: Practice and a Pre-Registered Test. 5D Library. https://doi.org/10.5281/zenodo.23836264
DOI: 10.5281/zenodo.23836264 · Read the monograph · 5DL-B-048 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026048,
author = {Fan, Yafu},
title = {Algorithmic Marketing, Multi-Touch Shapley Attribution and Stochastic Control of Acquisition Funnels},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.23836264},
url = {https://doi.org/10.5281/zenodo.23836264}
}Other chapter answers in this book
- Chapter 1 · Attribution Is Not Incrementality: Last Click, Potential Outcomes and Randomised Holdouts
- Chapter 2 · Shapley Values and Markov Removal Effects: Exact Allocations of an Inexact Question
- Chapter 3 · The Price of Peeking: Optional Stopping, Group-Sequential Designs and Always-Valid Inference
- Chapter 4 · Learning While Earning: Thompson Sampling and the Bias of Adaptive Experiments
- Chapter 5 · What Is a Customer Still Worth? BG/NBD, Gamma-Gamma and the Illusion of Rising Retention
- Chapter 6 · Diffusion and Spend: the Bass Model, the Dorfman–Steiner Condition and Whole-Life Budgets
- Chapter 7 · Reading the Classics: *Shi* and *Jie* in the *Sunzi*