5D Library › Algorithmic Marketing, Multi-Touch Shapley Attribution and Stochastic Control of Acquisition Funnels · 中文
Chapter 1 Attribution Is Not Incrementality: Last Click, Potential Outcomes and Randomised Holdouts
Two different questions
The “attribution” in a marketing report often answers two different questions at once. The first is a bookkeeping question: how should the credit for a conversion that has already happened be divided among the channels that appeared on its path? The second is a causal question: if we stopped spending on a channel next month, how many fewer conversions would there be? The first can be answered by any rule; the second requires comparing two worlds, the one in which we spend and the one in which we do not, and we only ever see one of them.
Statistics writes the second question in terms of potential outcomes (Holland 1986). For user , let be whether he converts when he sees a channel’s ad and whether he converts when he does not. The channel’s causal effect on him is , and its incrementality over the whole population is . Holland called this the fundamental problem of causal inference: a person can be in only one state, so only one of and is ever observed. Randomisation matters because, under random assignment, the treated and control groups have the same distribution in every other respect, and the difference of their average outcomes is an unbiased estimate of the average causal effect.
Rule-based attribution (last click, first click, linear, time decay, position-based) answers the first question. These rules are not wrong, since a bookkeeping rule is neither right nor wrong; the error is to take the bookkeeping result as the answer to the second question and to allocate budgets by it.
A fictional world whose truth is known
To compare how far attribution methods are from the truth, we need a world in which the truth is known. Real data provide no such world, so what follows uses a fictional one; it is the author’s model, with illustrative parameters.
Of 200,000 fictional users, 20 % have high purchase intent. There are four channels: search, social, display and email. Exposure to search and email depends strongly on intent (only those who want to buy search for the brand name or sit on the member list), social is only lightly targeted, and display is not targeted at all. Each user’s conversion probability equals a base probability set by intent (8 % for high intent, 0.5 % for low), plus the causal lift of each channel he sees (1.0, 3.0, 0.8 and 1.2 percentage points for search, social, display and email). Because the lifts add, each channel’s true incrementality is uniquely defined and can be computed directly from the model: the number exposed times the lift.
Calculation
# A FICTIONAL world of customer journeys whose true incrementality is known.
# 20 % of users have high purchase intent. Four channels: Search (S), Social (F),
# Display (D), Email (E). Exposure to S and E depends strongly on intent (selection);
# D is untargeted. Conversion probability = base(intent) + sum of the causal lifts
# of the channels seen (additive, so each channel's incremental effect is unique).
# Touch times: D and F early, S late, E anywhere; a conversion follows the last touch.
# Rule-based attribution (last, first, linear click) is compared with the truth.
import random
CH = "SFDE"
P_HIGH = 0.20
EXPO = {"S": (0.60, 0.10), "F": (0.35, 0.30), "D": (0.50, 0.50), "E": (0.60, 0.15)}
BASE = (0.08, 0.005) # (high intent, low intent)
LIFT = {"S": 0.010, "F": 0.030, "D": 0.008, "E": 0.012}
WHEN = {"S": (0.6, 1.0), "F": (0.0, 0.7), "D": (0.0, 0.5), "E": (0.2, 1.0)}
def user(rng, sel=1.0):
"""One journey: (intent, channels in time order, converted)."""
hi = rng.random() < P_HIGH
touches = []
for c in CH:
p_hi, p_lo = EXPO[c]
p = p_lo + sel * (p_hi - p_lo) if hi else p_lo # sel = 0: no selection
if rng.random() < p:
touches.append((rng.uniform(*WHEN[c]), c))
path = [c for _, c in sorted(touches)]
p_conv = BASE[0 if hi else 1] + sum(LIFT[c] for c in path)
return hi, path, rng.random() < p_conv
def truth(n, sel=1.0):
"""Expected incremental conversions per channel (exact, from the model)."""
out = {}
for c in CH:
p_hi, p_lo = EXPO[c]
expo = P_HIGH * (p_lo + sel * (p_hi - p_lo)) + (1 - P_HIGH) * p_lo
out[c] = n * expo * LIFT[c]
return out
def shares(d):
tot = sum(d.values())
return {c: d[c] / tot for c in CH}
def rules(data):
last, first, linear = ({c: 0.0 for c in CH} for _ in range(3))
for _, path, conv in data:
if conv and path:
last[path[-1]] += 1; first[path[0]] += 1
for c in path:
linear[c] += 1 / len(path)
return last, first, linear
if __name__ == "__main__":
n = 200_000
rng = random.Random(2026)
data = [user(rng) for _ in range(n)]
conv = sum(d[2] for d in data)
untouched = sum(d[2] for d in data if not d[1])
tr = truth(n)
base = n * (P_HIGH * BASE[0] + (1 - P_HIGH) * BASE[1])
print(f"{n:,} fictional users: {conv:,} conversions ({untouched:,} with no touch)")
print(f"expected: {base:,.0f} would happen without any ad,"
f" {sum(tr.values()):,.0f} are incremental")
last, first, linear = rules(data)
print("share of credit truth last click first click linear")
T, L, F, Li = shares(tr), shares(last), shares(first), shares(linear)
names = {"S": "Search", "F": "Social", "D": "Display", "E": "Email"}
for c in CH:
print(f" {names[c]:8s} {T[c]:6.1%} {L[c]:6.1%} {F[c]:6.1%}"
f" {Li[c]:6.1%}")
for lab, s in (("last click", L), ("first click", F), ("linear", Li)):
mae = sum(abs(s[c] - T[c]) for c in CH) / len(CH)
print(f" mean absolute error of {lab:11s} {mae:6.1%}")
credited = sum(last.values())
print(f"last click credits {credited:,.0f} conversions to ads;"
f" the ads caused about {sum(tr.values()):,.0f}")
# 200,000 fictional users: 7,676 conversions (377 with no touch)
# expected: 4,000 would happen without any ad, 3,636 are incremental
# share of credit truth last click first click linear
# Search 11.0% 38.5% 6.4% 20.3%
# Social 51.2% 21.0% 31.2% 27.4%
# Display 22.0% 16.1% 46.7% 30.4%
# Email 15.8% 24.5% 15.7% 22.0%
# mean absolute error of last click 18.1%
# mean absolute error of first click 12.3%
# mean absolute error of linear 11.9%
# last click credits 7,299 conversions to ads; the ads caused about 3,636The world has 7,676 conversions, of which about 4,000 would have happened without any advertising (high-intent users buy anyway); the ads caused only about 3,636. Last click credits 7,299 conversions to the ads, twice the true incrementality. The shares matter more: social’s true incremental share is 51.2 %, and last click gives it 21.0 %; search’s true share is 11.0 %, and last click gives it 38.5 %. There are two reasons, independent of each other. First, search usually comes at the end of a path (people search once they have decided to buy), so the last-touch rule naturally favours it. Second, search and email mainly reach people who were going to buy anyway, so they “appear” on paths much more often than they “cause” conversions. First click and linear do little better, with mean absolute errors of 12.3 and 11.9 percentage points (last click 18.1).
Note
The example illustrates a mechanism, not proportions. In the real world the targeting of each channel, its position on paths and its causal lift are unknown, and last click may overstate or understate any given channel. The program shows only this: as long as exposure is correlated with purchase intent, any rule that looks only at paths credits the ads with what intent did.
The algebra of selection bias
To write the intuition as a formula, let exposure to channel be . The observed difference “conversion rate of the exposed minus that of the unexposed” decomposes as
The second term is the extent to which the people who were shown ads would have been more likely to buy even without them. The more precise the targeting, the larger this term. Ad platforms’ targeting systems aim precisely at finding “the people most likely to convert”, so in digital advertising selection bias is the rule, not the exception. Rule-based attribution, like the Shapley and Markov methods of Chapter 2, uses only conditional expectations of the kind , from which the second term cannot be removed.
Evidence: the gap between experiments and observational methods
Evidence
Why advertising effects are hard to measure. Lewis and Rao analysed 25 large randomised advertising experiments run with major US retailers and brokerages, with about $2.8 million of advertising spend, most with more than a million unique users (Lewis and Rao 2015). They found that individual-level sales are extremely volatile relative to the per-capita cost of advertising (a coefficient of variation of about 10 is common), and the median width of the confidence interval for return on investment exceeded 100 percentage points; informative conclusions could require samples of more than ten million person-weeks. So even with a randomised experiment, the return of a single campaign is often poorly measured; without one, separating the effect of advertising from the effect of intent with observational data alone is harder still.
Evidence
Observational methods against large experiments. Gordon, Zettelmeyer, Bhargava and Chapsky used 15 US advertising experiments at Facebook (about 500 million user–experiment observations and 1.6 billion ad impressions) as a benchmark, and asked whether common observational methods — exact matching, stratification, propensity-score matching, regression adjustment and inverse-probability-weighted regression adjustment — could recover the experimental results (Gordon et al. 2019). Often they could not, even after conditioning on extensive demographic and behavioural variables; overall they mostly overestimated advertising effects, though in some cases they underestimated them; in half of the studies the estimated percentage lift in purchase outcomes was off by a factor of about three across all methods. Richer covariates usually moved the estimates closer to the experiment, but results were inconsistent across studies.
Evidence
Brand-keyword search ads. Blake, Nosko and Tadelis ran large-scale field experiments on paid search at eBay (Blake et al. 2015). For brand keywords, when the ads were switched off traffic moved almost entirely to the organic search results, and the ads had no measurable short-term benefit; for non-brand keywords, new and infrequent users were positively influenced, while the frequent users who account for most of the spending were barely affected. They estimated that the returns to paid search are a fraction of conventional non-experimental estimates, and that average returns on non-brand keywords were negative. This is the real-world version of “search comes at the end of the path and reaches people who were going to buy anyway”.
Measuring incrementality: holdouts and ghost ads
The most direct way to measure incrementality is a randomised holdout: a random subset of users is prevented from seeing a channel’s ads, and its conversion rate is compared with that of everyone else. It has three difficulties. First, it is noisy and, as Lewis and Rao showed, needs very large samples. Second, if the control group sees a substitute ad (a public-service ad, say), the platform’s delivery algorithm may show the substitute to different people from those who would have seen the real ad, and the two groups are no longer comparable. Third, the real ad reaches only part of the treated group (the platform decides through auctions and targeting); comparing only “those in the treated group who were actually exposed” with the whole control group reintroduces selection bias, while comparing the whole treated group with the whole control group (intention to treat) is unbiased but diluted by the many who would never be exposed, and so noisier still.
The ghost ads method of Johnson, Lewis and Nubbemeyer solves the last two problems (Johnson et al. 2017). In the control group, the ad system runs its auction and targeting logic as usual and records the moments at which “this user would have been shown the ad if he were in the treated group”, but actually shows something else. Exposed users in the treated group can then be compared with “ghost-exposed” users in the control group: both are selected by the same delivery logic and both exclude people who would never have been reached, so the comparison is unbiased and more precise, and no substitute ads need be paid for.
Summary
Attribution rules answer a bookkeeping question; incrementality answers a causal one. Exposure unrelated to purchase intent is necessary for the two to agree, but for last click it is not sufficient: in the untargeted version of the same fictional world, last click is still biased by path position. In a fictional world whose truth is known, last click credits the ads with twice their true incrementality and moves credit from early, untargeted channels to late, heavily targeted ones. Studies comparing observational methods with large randomised experiments show that, even with rich covariates, observational estimates are often off by a factor of several; experiments on brand-keyword search show that the return of an end-of-path channel can be close to zero. The causal benchmark is the randomised holdout or ghost-ad experiment, at the cost of very large samples.
Hypothesis · not established · Systematic overstatement of end-of-path channels
The author’s hypothesis (not established): among e-commerce advertisers who use last-click attribution and target search and email heavily, the share last click gives to brand search and email is systematically higher than the incremental share estimated by randomised holdouts. What result would show it to be wrong: in a pre-registered set of advertisers (for example at least 20, recruited by independent researchers), the 95 % confidence interval of the pooled estimate of “last-click share minus holdout incremental share” contains zero or lies entirely below zero.
References
Fan, Y. (2026). Algorithmic Marketing, Multi-Touch Shapley Attribution and Stochastic Control of Acquisition Funnels: From Thompson Sampling Bandits and Absorbing Markov Chains to BG/NBD Customer Lifetime Value, Read Beside the Strategic Dispositions of the Sunzi [算法营销、多触点沙普利归因与获客漏斗的随机控制] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23836264 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).