What does Attribution, Bandits and Customer Lifetime Value conclude, in numbers, about “Practice and a Pre-Registered Test”?

Answer: Chapter 8 gives a four-step checklist (attribution, experiments, lifetime value, budget) and a pre-registered simulation benchmark: on a synthetic multichannel data generator with known true increments, frozen by an independent third party, observational Shapley attribution is compared with last click on their error against the true incremental shares (single primary outcome the difference in mean absolute errors, smallest effect of interest 2 percentage points, 79 synthetic worlds for 90 % power, 90 planned, decision rule written as code). Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's models and hypotheses; all data in the book are fictional or simulated.

Scope of the numbers: Which advertising channel should a sale be credited to? When may an A/B test stop? What is a customer still worth? This book presents mathematical models for teaching and research and is not marketing, investment or legal advice. Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's models and hypotheses; all data in the book are fictional or simulated.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 79, 90

pilot (120 worlds, 20,000 users each): MAE last click 13.0%, Shapley 4.7%
   D = MAE(last) - MAE(Shapley): mean +8.28%, sd 5.48%; Shapley better in 112/120
   secondary, truth = normalised removal increments v*(N) - v*(N - c):
      mean D +8.02%, Shapley better in 112/120
SESOI 2 percentage points; 90 % power, two-sided alpha 0.05:
   79 worlds (normal approximation, sd from the pilot)
   simulated power at n = 79: 0.899 (4,000 resamples of the pilot differences)
   plus 10 % for failed or degenerate worlds, rounded up: plan 90 worlds
sensitivity (worlds)        SESOI: 1 pt     2 pt     3 pt
   sd as in the pilot               316      79      36
   sd 50 % higher                   711     178      79
Show the full program
# Sample size for the Chapter 8 pre-registered simulation benchmark. Unit: a synthetic
# world drawn from a generator frozen by an independent third party. In each world the
# TRUE credit of a channel is the Shapley value of the interventional game v*(S) =
# expected conversions if exactly the channels in S were switched on (known exactly
# from the generator; for additive worlds it equals each channel's incremental
# conversions). Primary outcome per world: D = MAE(last click) - MAE(observational
# Shapley), the mean absolute error of channel shares against the truth. Secondary
# (sensitivity): the same D against normalised removal increments v*(N) - v*(N - c).
# PILOT GENERATOR (a planning assumption, not the trial's generator): 3-5 channels,
# random targeting strength, lifts, exposure and an optional two-channel synergy.
import itertools, math, random, statistics

# --- helpers repeated from earlier programs, so that this one runs on its own ---
def shapley(players, v):
    n, phi = len(players), {}
    for i in players:
        rest, s = [p for p in players if p != i], 0.0
        for k in range(n):
            w = math.factorial(k) * math.factorial(n - k - 1) / math.factorial(n)
            for T in itertools.combinations(rest, k):
                s += w * (v(frozenset(T) | {i}) - v(frozenset(T)))
        phi[i] = s
    return phi
# --- end of repeated helpers ---

Z = statistics.NormalDist().inv_cdf

def world(rng):
    K = rng.choice((3, 4, 5)); ch = "ABCDE"[:K]; ph = rng.uniform(0.1, 0.3)
    sel = rng.random()
    lo = {c: rng.uniform(0.05, 0.5) for c in ch}
    hi = {c: min(0.95, lo[c] + sel * rng.uniform(0, 0.5)) for c in ch}
    lift = {c: rng.uniform(0.0, 0.03) for c in ch}
    when = {c: rng.random() for c in ch}                  # typical position in a path
    syn = (rng.sample(ch, 2), rng.uniform(0, 0.04)) if rng.random() < 0.5 else (None, 0)
    base = (rng.uniform(0.03, 0.10), rng.uniform(0.002, 0.01))
    return ch, ph, lo, hi, lift, when, syn, base

def v_star(Wd, S):
    ch, ph, lo, hi, lift, when, syn, base = Wd
    out = 0.0
    for w, e, b in ((ph, hi, base[0]), (1 - ph, lo, base[1])):
        x = b + sum(e[c] * lift[c] for c in S)
        if syn[0] and all(c in S for c in syn[0]):
            x += e[syn[0][0]] * e[syn[0][1]] * syn[1]
        out += w * x
    return out

def users(rng, Wd, n):
    ch, ph, lo, hi, lift, when, syn, base = Wd
    data = []
    for _ in range(n):
        h = rng.random() < ph; e = hi if h else lo
        seen = [c for c in ch if rng.random() < e[c]]
        path = sorted(seen, key=lambda c: when[c] + 0.3 * rng.random())
        p = base[0 if h else 1] + sum(lift[c] for c in seen)
        if syn[0] and all(c in seen for c in syn[0]):
            p += syn[1]
        data.append((path, rng.random() < p))
    return data

def mae(a, b, ch):
    return sum(abs(a[c] - b[c]) for c in ch) / len(ch)

def score(rng, n_users=20_000):
    Wd = world(rng); ch = Wd[0]
    tru = shapley(list(ch), lambda S: v_star(Wd, S) - v_star(Wd, ()))
    data = users(rng, Wd, n_users)
    last = {c: 0.0 for c in ch}
    for path, y in data:
        if y and path:
            last[path[-1]] += 1
    cnt, conv = {}, {}
    for path, y in data:
        S = frozenset(path); cnt[S] = cnt.get(S, 0) + 1; conv[S] = conv.get(S, 0) + y
    rate = {S: conv[S] / cnt[S] for S in cnt}
    v = lambda T: rate.get(frozenset(T), 0.0) - rate.get(frozenset(), 0.0)
    obs = {c: 0.0 for c in ch}
    for S, m in cnt.items():
        if S:
            for c, x in shapley(sorted(S), v).items():
                obs[c] += m * x
    norm = lambda d: {c: d[c] / sum(d.values()) if sum(d.values()) else 1 / len(ch)
                      for c in ch}
    t, l, s = norm({c: max(tru[c], 0.0) for c in ch}), norm(last), norm(obs)
    full = v_star(Wd, ch)                     # alternative truth: removal increments
    alt = norm({c: max(full - v_star(Wd, [d for d in ch if d != c]), 0.0) for c in ch})
    return mae(l, t, ch), mae(s, t, ch), mae(l, alt, ch) - mae(s, alt, ch)

if __name__ == "__main__":
    rng = random.Random(4800); n0 = 120
    R = [score(rng) for _ in range(n0)]
    D = [a - b for a, b, _ in R]
    m, sd = statistics.fmean(D), statistics.stdev(D)
    print(f"pilot ({n0} worlds, 20,000 users each): MAE last click"
          f" {statistics.fmean(r[0] for r in R):.1%},"
          f" Shapley {statistics.fmean(r[1] for r in R):.1%}")
    print(f"   D = MAE(last) - MAE(Shapley): mean {m:+.2%}, sd {sd:.2%};"
          f" Shapley better in {sum(d > 0 for d in D)}/{n0}")
    Da = [r[2] for r in R]
    print(f"   secondary, truth = normalised removal increments v*(N) - v*(N - c):"
          f"\n      mean D {statistics.fmean(Da):+.2%}, Shapley better in"
          f" {sum(d > 0 for d in Da)}/{n0}")
    sesoi = 0.02
    n = math.ceil(((Z(0.975) + Z(0.90)) * sd / sesoi) ** 2)
    print(f"SESOI 2 percentage points; 90 % power, two-sided alpha 0.05:")
    print(f"   {n} worlds (normal approximation, sd from the pilot)")
    hits, rs = 0, random.Random(9)
    for _ in range(4000):                                 # resampling check of power
        s = [d - m + sesoi for d in rs.choices(D, k=n)]   # pilot shape, mean = SESOI
        se = statistics.stdev(s) / math.sqrt(n)
        hits += statistics.fmean(s) - Z(0.975) * se > 0
    print(f"   simulated power at n = {n}: {hits / 4000:.3f} (4,000 resamples of the"
          f" pilot differences)")
    plan = math.ceil(n * 1.10 / 10) * 10
    print(f"   plus 10 % for failed or degenerate worlds, rounded up: plan {plan} worlds")
    print("sensitivity (worlds)        SESOI: 1 pt     2 pt     3 pt")
    for lab, f in (("sd as in the pilot", 1.0), ("sd 50 % higher", 1.5)):
        row = [math.ceil(((Z(0.975) + Z(0.90)) * sd * f / e) ** 2) for e in
               (0.01, 0.02, 0.03)]
        print(f"   {lab:28s}" + "".join(f"{x:8,d}" for x in row))

Single source and citation

Fan, Y. (2026). Algorithmic Marketing, Multi-Touch Shapley Attribution and Stochastic Control of Acquisition Funnels (Version 1.0.0), Chapter 8: Practice and a Pre-Registered Test. 5D Library. https://doi.org/10.5281/zenodo.23836264

DOI: 10.5281/zenodo.23836264 · Read the monograph · 5DL-B-048 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026048,
  author    = {Fan, Yafu},
  title     = {Algorithmic Marketing, Multi-Touch Shapley Attribution and Stochastic Control of Acquisition Funnels},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.23836264},
  url       = {https://doi.org/10.5281/zenodo.23836264}
}

Other chapter answers in this book