5D Library › 算法营销、多触点沙普利归因与获客漏斗的随机控制 · English
第八章 实践与预注册检验
注
本章的核查表与研究方案用于研究与自我检查,不是营销、投资或法律建议。 核查表不能告诉你该在哪个渠道花多少钱,只能帮助你在做决定之前把假设写清楚。书中的用户、路径、测试、客户与花费都是虚构或模拟的;任何真实的广告投放都可能亏损。
第一部分 四步核查表
第一至六章可以浓缩为一份四步核查表,用于审视一个季度的营销报表。表中的界限(例如「末次点击的份额落在持出实验置信区间之外时标记」)是使用者应当事先书面写定的规则的例子,不是推荐的数值。
- 归因与增量。 报表中的渠道功劳来自什么规则?至少对主要渠道,有没有随机持出或幽灵广告实验的增量估计?归因份额是否落在实验估计的置信区间之内(第一、二章)?
- 实验的停止规则。 每个 A/B 测试的样本量与停止规则是否事先写定?如果中途查看过,查看了几次,是否用了群序贯或始终有效的方法?赌博机分配的实验,报告的效果是否做了倾向加权(第三、四章)?
- 客户价值。 生命周期价值是用第一年留存率连乘算出来的,还是用了考虑异质性的模型?平均客单价与购买频率是否相关(第五章)?
- 预算的边际。 每个渠道的预算是按平均回报(总收入/总花费)还是按边际回报(多花 1 千元多带来多少)决定的?平均回报高的渠道,边际回报可能已经低于成本(第六章)。
下面的程序把核查表用于一个虚构的季度:40 万个用户来自第一章的虚构世界,每个渠道都对一个独立随机抽取的 10% 用户做持出(幽灵广告式:在本会被投放的人中压下广告);三个 A/B 测试的查看记录;一个用第一年留存率连乘算出的生命周期价值;以及三个渠道在一条虚构的凹响应曲线上的平均与边际回报。
推导
# The four-step check applied to a FICTIONAL quarter. 400,000 users from the world of
# p01; for each channel, an independent random 10 % of users are held out (the ad is
# suppressed, ghost-ad style, so the holdout is defined among would-be-exposed users).
# Step 1: last-click credit against the holdout estimate of incremental conversions.
# Step 2: a test log with the number of looks at the data, against p04's error rates.
# Step 3: a lifetime claim built by multiplying first-year retention (against p09).
# Step 4: average against marginal return per channel (concave response, fictional).
import math, random
# --- helpers repeated from earlier programs, so that this one runs on its own ---
CH = "SFDE"
P_HIGH = 0.20
EXPO = {"S": (0.60, 0.10), "F": (0.35, 0.30), "D": (0.50, 0.50), "E": (0.60, 0.15)}
BASE = (0.08, 0.005) # (high intent, low intent)
LIFT = {"S": 0.010, "F": 0.030, "D": 0.008, "E": 0.012}
WHEN = {"S": (0.6, 1.0), "F": (0.0, 0.7), "D": (0.0, 0.5), "E": (0.2, 1.0)}
def truth(n, sel=1.0):
"""Expected incremental conversions per channel (exact, from the model)."""
out = {}
for c in CH:
p_hi, p_lo = EXPO[c]
expo = P_HIGH * (p_lo + sel * (p_hi - p_lo)) + (1 - P_HIGH) * p_lo
out[c] = n * expo * LIFT[c]
return out
def stay_probs(bounds, h=0.1, drift=0.0):
"""P(|S_j| < bounds[j-1] for all j <= k), k = 1..K, S a Gaussian random walk with
N(drift, 1) steps: Simpson's rule on a grid spanning each continuation interval."""
phi = lambda x: math.exp(-x * x / 2) / math.sqrt(2 * math.pi)
def grid(b):
n = 2 * max(1, math.ceil(b / h)); d = 2 * b / n
w = [(1 if i in (0, n) else 4 if i % 2 else 2) * d / 3 for i in range(n + 1)]
return [-b + i * d for i in range(n + 1)], w
xs, w = grid(bounds[0]); f = [phi(x - drift) for x in xs]
out = [sum(a * b for a, b in zip(w, f))]
for b in bounds[1:]:
ys, v = grid(b)
g = [sum(wi * fi * phi(y - x - drift) for x, wi, fi in zip(xs, w, f)
if abs(y - x - drift) < 8.5)
for y in ys]
xs, w, f = ys, v, g
out.append(sum(a * b for a, b in zip(w, f)))
return out
def sbg_surv(t, al, be):
return math.exp(math.lgamma(be + t) - math.lgamma(be) + math.lgamma(al + be)
- math.lgamma(al + be + t))
# --- end of repeated helpers ---
def quarter(rng, n):
rows = []
for _ in range(n):
hi = rng.random() < P_HIGH
elig = [c for c in CH if rng.random() < EXPO[c][0 if hi else 1]]
held = {c for c in CH if rng.random() < 0.10}
seen = sorted((rng.uniform(*WHEN[c]), c) for c in elig if c not in held)
path = [c for _, c in seen]
conv = rng.random() < BASE[0 if hi else 1] + sum(LIFT[c] for c in path)
rows.append((set(elig), held, path, conv))
return rows
if __name__ == "__main__":
n = 400_000; rows = quarter(random.Random(48), n)
names = {"S": "Search", "F": "Social", "D": "Display", "E": "Email"}
print("1 attribution vs incrementality (flag if last-click credit is outside the")
print(" 95 % interval of the holdout estimate); conversions in the quarter")
tr = truth(n)
for c in CH:
e = [r for r in rows if c in r[0]]
t = [r for r in e if c not in r[1]]; h = [r for r in e if c in r[1]]
pt, ph = sum(r[3] for r in t) / len(t), sum(r[3] for r in h) / len(h)
se = math.sqrt(pt * (1 - pt) / len(t) + ph * (1 - ph) / len(h))
inc, lo, hi = (pt - ph) * len(t), (pt - ph - 1.96 * se) * len(t), (
pt - ph + 1.96 * se) * len(t)
lc = sum(1 for r in rows if r[3] and r[2] and r[2][-1] == c)
flag = "FLAG" if not lo <= lc <= hi else "ok"
print(f" {names[c]:8s} last click {lc:5,d}; holdout {inc:5,.0f} ({lo:5,.0f}"
f" to {hi:5,.0f}); truth {tr[c] * 0.9:5,.0f} -> {flag}")
print("2 test log (flag if the stopping rule was not fixed in advance)")
z = 1.959964; st = stay_probs([z * math.sqrt(k) for k in range(1, 31)])
for name, looks, fixed in (("landing page B", 1, True), ("subject line", 14, False),
("price badge", 30, False)):
a = 1 - st[looks - 1]
yn = "yes" if fixed else "no "
print(f" {name:15s} looks {looks:2d}, rule fixed {yn}:"
f" actual false-positive rate {a:.3f} -> {'ok' if fixed else 'FLAG'}")
print("3 lifetime claim (flag if a heterogeneous-cohort model differs by > 20 %)")
al, be, r1, d = 0.70407, 1.18204, 0.631, 0.10 # sBG fit of p09
ls = sum(sbg_surv(t, al, be) / (1 + d) ** t for t in range(400))
ln = sum((r1 / (1 + d)) ** t for t in range(400))
print(f" naive {ln:.2f} years, sBG {ls:.2f} years ({ls / ln - 1:+.0%})"
f" -> {'FLAG' if abs(ls / ln - 1) > 0.2 else 'ok'}")
print("4 budget: average and marginal return per extra 1,000 (flag if marginal < 1)")
for c, spend, k in (("S", 40_000, 9.0), ("F", 60_000, 5.0), ("E", 60_000, 3.0),
("D", 30_000, 1.2)):
rev = lambda a: k * 20_000 * math.log(1 + a / 20_000) # fictional response
avg = rev(spend) / spend; marg = (rev(spend + 1000) - rev(spend)) / 1000
print(f" {names[c]:8s} spend {spend:6,d}: average {avg:4.2f},"
f" marginal {marg:4.2f}"
f" -> {'FLAG' if marg < 1 else 'ok'}")
# 1 attribution vs incrementality (flag if last-click credit is outside the
# 95 % interval of the holdout estimate); conversions in the quarter
# Search last click 4,889; holdout 766 ( 344 to 1,187); truth 720 -> FLAG
# Social last click 2,956; holdout 3,111 (2,734 to 3,488); truth 3,348 -> ok
# Display last click 2,231; holdout 1,332 ( 855 to 1,809); truth 1,440 -> FLAG
# Email last click 3,488; holdout 1,384 ( 958 to 1,810); truth 1,037 -> FLAG
# 2 test log (flag if the stopping rule was not fixed in advance)
# landing page B looks 1, rule fixed yes: actual false-positive rate 0.050 -> ok
# subject line looks 14, rule fixed no : actual false-positive rate 0.220 -> FLAG
# price badge looks 30, rule fixed no : actual false-positive rate 0.280 -> FLAG
# 3 lifetime claim (flag if a heterogeneous-cohort model differs by > 20 %)
# naive 2.35 years, sBG 3.69 years (+58%) -> FLAG
# 4 budget: average and marginal return per extra 1,000 (flag if marginal < 1)
# Search spend 40,000: average 4.94, marginal 2.98 -> ok
# Social spend 60,000: average 2.31, marginal 1.24 -> ok
# Email spend 60,000: average 1.39, marginal 0.75 -> FLAG
# Display spend 30,000: average 0.73, marginal 0.48 -> FLAG第一步标记了三个渠道:搜索的末次点击功劳是 4,889 次转化,持出实验估计的增量只有 766 次(95% 区间 344 到 1,187),真值为 720 次;邮件为 3,488 对 1,384;展示为 2,231 对 1,332。社交的末次点击功劳 2,956 次落在实验区间(2,734 到 3,488)之内,没有标记——这不说明末次点击对社交「准确」,只是碰巧接近。第二步标记了两个测试:一个查看了 14 次、一个查看了 30 次,都没有事先写定停止规则,实际第一类错误率分别为 22.0% 与 28.0%。第三步标记:连乘得到的期望存续年数为 2.35 年,sBG 为 3.69 年,相差 58%。第四步标记了两个渠道:展示的平均回报本来就低于 1(0.73),边际回报更低(0.48);邮件的平均回报为 1.39,看起来每花 1 元能带回 1.39 元,但边际回报只有 0.75,再加投就是亏的——这正是只看平均回报会漏掉的情形。搜索的平均回报 4.94,边际回报 2.98 仍高于 1,这条虚构曲线上还有加投的空间。这个例子的教训是:四步各看一个侧面,归因报表、测试结论、客户价值与预算边际可以各自出错,而且错误的方向不一定相同。
第二部分 预注册的模拟基准研究
问题与设计选择
在一个由独立第三方构建并冻结、已知真实增量的合成多渠道数据生成器上,观测数据上的沙普利值归因,对真实增量份额的误差是否小于末次点击,且差别至少达到最小关注效应?
作者的框架提出两个方向:(a) 在已知真相的合成数据上比较归因方法;(b) 在真实广告主那里做随机持出实验。本书选 (a),并作三点说明。第一,为什么不选 (b):真实广告主的持出实验检验的是某个广告活动的增量,而不是归因方法本身;要比较方法,就需要知道每个渠道的真实增量份额,这在真实数据中只能靠对每个渠道分别做持出实验来近似,而第一章已说明单个持出实验的噪声往往很大 (Lewis and Rao 2015);此外,真实广告主的数据难免与作者企业的业务发生关联,增加利益冲突。第二,(a) 只检验方法在已知世界中的表现:生成器决定了结果,结论不能直接推及任何真实的广告市场,见下文「不能说明什么」。第三,由独立第三方实施:生成器的结构、参数分布与随机种子由与作者无关的研究者设计并封存,作者只提交两种归因方法的代码。
方案
| 要素 | 规定 |
|---|---|
| 生成器 | 由第三方设计、在登记方案时公开结构与参数分布的合成数据生成器:每个「世界」有 3–5 个渠道,曝光对潜在购买意向的依赖强度(定向)、各渠道的因果提升、路径中的典型位置与是否存在两渠道协同均从登记的分布中随机抽取;用户数每个世界 20,000 |
| 真实增量 | 对每个世界,用生成器的因果结构精确计算干预性特征函数 (只开 中渠道时的期望转化),真实份额定义为 的沙普利值归一化;在可加的世界中,它等于各渠道的增量 |
| 两种方法 | X:观测沙普利值(第二章程序 p02 的做法, 为曝光集合恰为 的用户的转化率);Y:末次点击。两者只看到路径与转化,看不到意向与 |
| 设盲 | 方法代码以哈希值登记;第三方把真实份额文件加密封存,运行与分析人员在结果锁定前不接触;作者不接触种子与生成器参数 |
| 主要结局 | 每个世界的 (以百分点计的渠道份额平均绝对误差之差),对世界取平均;95% 置信区间用对世界的自助法(10,000 次重抽)计算 |
| 预先规定的限定语 | X 自身的平均绝对误差是否超过 5 个百分点;若是,结论后附「比末次点击更接近,但离增量真相仍远」 |
| 次要结局 | 敏感性分析:改用另一种真相定义——归一化的单渠道移除增量 ——重算 ;马尔可夫移除效应的平均绝对误差;按定向强度与是否有协同分组的误差;记在广告名下的总转化与真实总增量之比;均为描述性或探索性分析 |
主要结局只设一项,选「平均绝对误差之差」,理由是:作者框架的核心主张是「沙普利值优于末次点击」,这一项直接对应;马尔可夫方法作为次要结局,是因为第二章已显示它在协同世界中的表现取决于生成器的结构,把它与沙普利值并列为共同主要结局需要多重比较校正,而且会把研究的问题从「是否优于末次点击」变成「哪种分配公式最好」。限定语的作用是防止一个「沙普利值显著更好」的结论被误读为「沙普利值足够好」:第二章已经说明,在有定向时它离增量仍然很远。
防止数据窥探
只有一组预先固定的方法实现与一项主要结局,作者不在生成器上调参;生成器结构、参数分布、方法代码与分析代码在运行之前公开登记,代码的哈希值随方案一起登记 (Nosek et al. 2018);全部运行与分析由第三方执行;次要与探索性结果明确标注,不能用来替代主要结论。第 33 本讨论过这些做法的统计原理。
样本量
样本量需要对差值 的标准差作出假设。本书用一个试点生成器来设定这个假设:它是作者自己写的玩具生成器,结构与上表相似,不是研究将要使用的生成器。
推导
# Sample size for the Chapter 8 pre-registered simulation benchmark. Unit: a synthetic
# world drawn from a generator frozen by an independent third party. In each world the
# TRUE credit of a channel is the Shapley value of the interventional game v*(S) =
# expected conversions if exactly the channels in S were switched on (known exactly
# from the generator; for additive worlds it equals each channel's incremental
# conversions). Primary outcome per world: D = MAE(last click) - MAE(observational
# Shapley), the mean absolute error of channel shares against the truth. Secondary
# (sensitivity): the same D against normalised removal increments v*(N) - v*(N - c).
# PILOT GENERATOR (a planning assumption, not the trial's generator): 3-5 channels,
# random targeting strength, lifts, exposure and an optional two-channel synergy.
import itertools, math, random, statistics
# --- helpers repeated from earlier programs, so that this one runs on its own ---
def shapley(players, v):
n, phi = len(players), {}
for i in players:
rest, s = [p for p in players if p != i], 0.0
for k in range(n):
w = math.factorial(k) * math.factorial(n - k - 1) / math.factorial(n)
for T in itertools.combinations(rest, k):
s += w * (v(frozenset(T) | {i}) - v(frozenset(T)))
phi[i] = s
return phi
# --- end of repeated helpers ---
Z = statistics.NormalDist().inv_cdf
def world(rng):
K = rng.choice((3, 4, 5)); ch = "ABCDE"[:K]; ph = rng.uniform(0.1, 0.3)
sel = rng.random()
lo = {c: rng.uniform(0.05, 0.5) for c in ch}
hi = {c: min(0.95, lo[c] + sel * rng.uniform(0, 0.5)) for c in ch}
lift = {c: rng.uniform(0.0, 0.03) for c in ch}
when = {c: rng.random() for c in ch} # typical position in a path
syn = (rng.sample(ch, 2), rng.uniform(0, 0.04)) if rng.random() < 0.5 else (None, 0)
base = (rng.uniform(0.03, 0.10), rng.uniform(0.002, 0.01))
return ch, ph, lo, hi, lift, when, syn, base
def v_star(Wd, S):
ch, ph, lo, hi, lift, when, syn, base = Wd
out = 0.0
for w, e, b in ((ph, hi, base[0]), (1 - ph, lo, base[1])):
x = b + sum(e[c] * lift[c] for c in S)
if syn[0] and all(c in S for c in syn[0]):
x += e[syn[0][0]] * e[syn[0][1]] * syn[1]
out += w * x
return out
def users(rng, Wd, n):
ch, ph, lo, hi, lift, when, syn, base = Wd
data = []
for _ in range(n):
h = rng.random() < ph; e = hi if h else lo
seen = [c for c in ch if rng.random() < e[c]]
path = sorted(seen, key=lambda c: when[c] + 0.3 * rng.random())
p = base[0 if h else 1] + sum(lift[c] for c in seen)
if syn[0] and all(c in seen for c in syn[0]):
p += syn[1]
data.append((path, rng.random() < p))
return data
def mae(a, b, ch):
return sum(abs(a[c] - b[c]) for c in ch) / len(ch)
def score(rng, n_users=20_000):
Wd = world(rng); ch = Wd[0]
tru = shapley(list(ch), lambda S: v_star(Wd, S) - v_star(Wd, ()))
data = users(rng, Wd, n_users)
last = {c: 0.0 for c in ch}
for path, y in data:
if y and path:
last[path[-1]] += 1
cnt, conv = {}, {}
for path, y in data:
S = frozenset(path); cnt[S] = cnt.get(S, 0) + 1; conv[S] = conv.get(S, 0) + y
rate = {S: conv[S] / cnt[S] for S in cnt}
v = lambda T: rate.get(frozenset(T), 0.0) - rate.get(frozenset(), 0.0)
obs = {c: 0.0 for c in ch}
for S, m in cnt.items():
if S:
for c, x in shapley(sorted(S), v).items():
obs[c] += m * x
norm = lambda d: {c: d[c] / sum(d.values()) if sum(d.values()) else 1 / len(ch)
for c in ch}
t, l, s = norm({c: max(tru[c], 0.0) for c in ch}), norm(last), norm(obs)
full = v_star(Wd, ch) # alternative truth: removal increments
alt = norm({c: max(full - v_star(Wd, [d for d in ch if d != c]), 0.0) for c in ch})
return mae(l, t, ch), mae(s, t, ch), mae(l, alt, ch) - mae(s, alt, ch)
if __name__ == "__main__":
rng = random.Random(4800); n0 = 120
R = [score(rng) for _ in range(n0)]
D = [a - b for a, b, _ in R]
m, sd = statistics.fmean(D), statistics.stdev(D)
print(f"pilot ({n0} worlds, 20,000 users each): MAE last click"
f" {statistics.fmean(r[0] for r in R):.1%},"
f" Shapley {statistics.fmean(r[1] for r in R):.1%}")
print(f" D = MAE(last) - MAE(Shapley): mean {m:+.2%}, sd {sd:.2%};"
f" Shapley better in {sum(d > 0 for d in D)}/{n0}")
Da = [r[2] for r in R]
print(f" secondary, truth = normalised removal increments v*(N) - v*(N - c):"
f"\n mean D {statistics.fmean(Da):+.2%}, Shapley better in"
f" {sum(d > 0 for d in Da)}/{n0}")
sesoi = 0.02
n = math.ceil(((Z(0.975) + Z(0.90)) * sd / sesoi) ** 2)
print(f"SESOI 2 percentage points; 90 % power, two-sided alpha 0.05:")
print(f" {n} worlds (normal approximation, sd from the pilot)")
hits, rs = 0, random.Random(9)
for _ in range(4000): # resampling check of power
s = [d - m + sesoi for d in rs.choices(D, k=n)] # pilot shape, mean = SESOI
se = statistics.stdev(s) / math.sqrt(n)
hits += statistics.fmean(s) - Z(0.975) * se > 0
print(f" simulated power at n = {n}: {hits / 4000:.3f} (4,000 resamples of the"
f" pilot differences)")
plan = math.ceil(n * 1.10 / 10) * 10
print(f" plus 10 % for failed or degenerate worlds, rounded up: plan {plan} worlds")
print("sensitivity (worlds) SESOI: 1 pt 2 pt 3 pt")
for lab, f in (("sd as in the pilot", 1.0), ("sd 50 % higher", 1.5)):
row = [math.ceil(((Z(0.975) + Z(0.90)) * sd * f / e) ** 2) for e in
(0.01, 0.02, 0.03)]
print(f" {lab:28s}" + "".join(f"{x:8,d}" for x in row))
# pilot (120 worlds, 20,000 users each): MAE last click 13.0%, Shapley 4.7%
# D = MAE(last) - MAE(Shapley): mean +8.28%, sd 5.48%; Shapley better in 112/120
# secondary, truth = normalised removal increments v*(N) - v*(N - c):
# mean D +8.02%, Shapley better in 112/120
# SESOI 2 percentage points; 90 % power, two-sided alpha 0.05:
# 79 worlds (normal approximation, sd from the pilot)
# simulated power at n = 79: 0.899 (4,000 resamples of the pilot differences)
# plus 10 % for failed or degenerate worlds, rounded up: plan 90 worlds
# sensitivity (worlds) SESOI: 1 pt 2 pt 3 pt
# sd as in the pilot 316 79 36
# sd 50 % higher 711 178 79试点中(120 个世界),末次点击的平均绝对误差为 13.0 个百分点,观测沙普利值为 4.7 个百分点; 的平均值为 +8.28 个百分点,标准差 5.48 个百分点,沙普利值在 112 个世界中更好;改用「归一化的单渠道移除增量」作真相时(次要结局), 的平均值为 +8.02 个百分点。这些数字是规划假设,不是证据:试点生成器由作者构建,其中一半的世界没有协同、定向强度在 0 到 1 之间均匀抽取,独立生成器中的差别可能小得多,甚至为负。
最小关注效应取 2 个百分点。换算成预算:按份额分配预算时,放错的预算比例是 ,所以 个渠道中 MAE 每差 1 个百分点,约多放错 个百分点的预算; 意味着末次点击比沙普利值多放错约 4% 的预算(四个渠道;三到五个渠道时为 3%–5%)。小于这个差别,就不值得为沙普利值增加的计算与解释成本改变做法。在双侧 、90% 把握度下,正态近似需要 个世界;从试点差值重抽、把均值平移到最小关注效应的模拟中,79 个世界的把握度为 0.899,与正态近似一致。考虑 10% 的世界因生成失败或退化(例如没有转化、某渠道无曝光)被排除,计划 90 个世界。敏感性表显示,最小关注效应若取 1 个百分点,需要 316 个世界;标准差若比试点高 50%,2 个百分点的最小关注效应需要 178 个世界。合成世界的成本很低,样本量本身不是瓶颈,瓶颈是事先锁定生成器与方法。
按第 33 本第六章的结果,「90% 把握度」指的是在真实效应恰好等于最小关注效应时,得出「沙普利值更接近」(置信区间下限大于 0)这类结论的概率约为 90%;在这种情况下,最强的结论「至少好出最小关注效应」只有约 2.5% 的机会出现。
判定规则
推导
# Decision rule, fixed before the third party runs the frozen generator. D = MAE(last
# click) - MAE(observational Shapley), in percentage points of channel share, averaged
# over worlds, with a 95 % CI from a bootstrap over worlds (10,000 resamples). SESOI
# 2 points. Gates, checked first: more than 10 % of worlds failed or degenerate; the
# independent re-run differing on more than 0.1 % of worlds; the generator's truth
# file opened before the results were locked -> NOT INTERPRETABLE.
# Pre-specified qualifier: Shapley's own MAE above 5 points ("better than last click,
# still far from the incremental truth").
SESOI = 2.0
def verdict(d, lo, hi, mae_sh, failed=0.02, mismatch=0.0, unblinded=False):
for bad, why in ((failed > 0.10, f"{failed:.0%} of worlds failed (limit 10 %)"),
(mismatch > 0.001, f"re-run differs on {mismatch:.2%}"
" (limit 0.1 %)"),
(unblinded, "truth file opened before lock")):
if bad:
return f"D = {d:+.1f}: NOT INTERPRETABLE - {why}"
if lo >= SESOI: v = "Shapley is closer to the truth by at least the SESOI"
elif lo > 0: v = "Shapley is closer; not shown to reach the SESOI"
elif hi < 0: v = "last click is closer to the truth"
elif lo >= -SESOI and hi <= SESOI: v = "no difference worth acting on"
else: v = "unclear: the interval is too wide to decide"
if mae_sh > 5.0:
v += f"\n qualifier: Shapley's own error is {mae_sh:.1f} points (limit 5)"
return f"D = {d:+.1f} ({lo:+.1f} to {hi:+.1f}): {v}"
for case in ((8.3, 7.1, 9.5, 4.7), (3.0, 1.2, 4.8, 3.9), (0.4, -0.9, 1.7, 9.0),
(-2.5, -3.9, -1.1, 11.0), (1.0, -2.6, 4.6, 6.0), (6.0, 4.4, 7.6, 8.2),
(8.0, 6.8, 9.2, 4.0, 0.14), (8.0, 6.8, 9.2, 4.0, 0.02, 0.0, True)):
print(verdict(*case))
print("secondary outcomes (Markov removal-effect MAE, MAE by targeting strength and")
print(" synergy, total credited against total incremental conversions) are reported")
print(" with intervals; they never decide the verdict")
# D = +8.3 (+7.1 to +9.5): Shapley is closer to the truth by at least the SESOI
# D = +3.0 (+1.2 to +4.8): Shapley is closer; not shown to reach the SESOI
# D = +0.4 (-0.9 to +1.7): no difference worth acting on
# qualifier: Shapley's own error is 9.0 points (limit 5)
# D = -2.5 (-3.9 to -1.1): last click is closer to the truth
# qualifier: Shapley's own error is 11.0 points (limit 5)
# D = +1.0 (-2.6 to +4.6): unclear: the interval is too wide to decide
# qualifier: Shapley's own error is 6.0 points (limit 5)
# D = +6.0 (+4.4 to +7.6): Shapley is closer to the truth by at least the SESOI
# qualifier: Shapley's own error is 8.2 points (limit 5)
# D = +8.0: NOT INTERPRETABLE - 14% of worlds failed (limit 10 %)
# D = +8.0: NOT INTERPRETABLE - truth file opened before lock
# secondary outcomes (Markov removal-effect MAE, MAE by targeting strength and
# synergy, total credited against total incremental conversions) are reported
# with intervals; they never decide the verdict示例结果是编造的。前置条件中任何一项不满足(超过 10% 的世界失败或退化、独立重跑与登记代码的结果在超过 0.1% 的世界上不一致、结果锁定前打开了真实份额文件),无论主要结局如何,报告都写「不可解释」。只有当置信区间下限不低于 2 个百分点时,报告才写「沙普利值至少好出最小关注效应」;下限高于 0 但低于 2 时写「沙普利值更接近,但幅度未确定」;上限低于 0 时写「末次点击更接近」;区间落在 ±2 之内时写「没有值得据此改变做法的差别」;其余情形写「无法判断」。限定语不改变结论的类别,只是附在后面:例如第六个示例中,沙普利值至少好出最小关注效应,但它自身的误差仍有 8.2 个百分点,报告必须同时写明这一点。
这项研究不能说明什么
第一,生成器不是真实市场:结论只对生成器中的定向、协同与路径结构成立,真实的选择偏差可能更强、更复杂,也可能随时间变化。第二,生成器的设计决定了结果;方案登记了参数分布,但无法覆盖所有形式的定向与交互。第三,「真相」的定义本身把沙普利值的平分规则写了进去:真实份额是干预性 的沙普利值,协同项按定义平分;在不定向的世界里,观测沙普利值在期望上恰好等于这个「真相」,而末次点击不等。所以主要结局在很大程度上取决于生成器中定向强度与路径位置的分布,这也是加入另一种真相定义作敏感性分析的原因。第四,它只比较一种沙普利值实现与末次点击,不检验其他沙普利变体(例如反事实调整的沙普利值 (Singal et al. 2022))或基于模型的归因。第五,即使沙普利值在合成世界中显著更好,也不说明它在真实数据中给出因果增量;第一、二章已说明,在有定向时它不能。第六,它不检验第三至六章的任何模型,那些需要各自的检验。第七,试点中的大效应来自作者自己的生成器,这正是需要独立生成器与预注册的原因;如果独立研究得到「没有值得据此改变做法的差别」,那就是这项研究的结论。第八,它不能代替真实广告主的随机持出实验;在任何具体的投放决策中,增量仍要靠实验测量。
伦理与利益冲突
这项研究不涉及人体受试者,不使用任何真实用户、广告主或客户的数据,不需要伦理委员会审批;但方案、生成器、方法代码与分析代码须在运行之前公开登记。无论结果是正是负,结果、代码与全部世界的汇总数据都将公开。为限制利益的影响:方案与代码在第三方运行前公开登记,作者不接触种子、生成器参数与真实份额文件,研究不使用作者企业或其客户的任何数据,无论结果如何都发表。
利益披露
作者以 5D Library 的著作(包括本书)作为其社群课程、升维牌卡(5D Ascension Cards,Dimensional Shift Card Decks)与 Yafu AI 知识库的方法论基础,因此与本书主题有商业利益关系。本书不推销任何课程、牌卡或服务;第八章的试验设计为由独立于作者企业的研究人员实施和分析,不从作者的社群、学员或客户中招募;书中所有代码与研究方案均公开,供任何人检验。作者创办并经营 Asia Influencer X 与 Yafu Global,从事网红营销、直播电商与数字营销(包括广告投放、效果归因与获客),工作中直接使用本书讨论的归因与实验方法,因此与本书主题有直接商业利益;本书不使用这两家公司或其客户的数据作为证据,也不推销其服务。
本章小结
四步核查表从归因与增量、实验的停止规则、客户价值与预算的边际四个方面审视一个季度的营销报表;虚构示例显示,末次点击可以把一个渠道的增量高估六倍,未写定停止规则的测试实际错误率可达 28%,留存率连乘与异质性模型相差 58%,一个平均回报 1.39 的渠道边际回报只有 0.75,已低于成本。预注册的模拟基准研究在独立第三方冻结、已知真实增量的生成器上,以 90 个合成世界比较观测沙普利值与末次点击对真实增量份额的误差,唯一主要结局为二者平均绝对误差之差,最小关注效应 2 个百分点,90% 把握度,判定规则写成代码,并预先规定了「离增量真相仍远」的限定语;无论结果如何都发表。它只检验方法在一个已知真相的模拟世界中的表现,不能证明本书的其他模型,也不能代替真实的增量实验。
参考文献
Fan, Y. (2026). Algorithmic Marketing, Multi-Touch Shapley Attribution and Stochastic Control of Acquisition Funnels: From Thompson Sampling Bandits and Absorbing Markov Chains to BG/NBD Customer Lifetime Value, Read Beside the Strategic Dispositions of the Sunzi [算法营销、多触点沙普利归因与获客漏斗的随机控制] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23836264 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).