What does Attribution, Bandits and Customer Lifetime Value conclude, in numbers, about “Attribution Is Not Incrementality: Last Click, Potential Outcomes and Randomised Holdouts”?

Answer: The world has 7,676 conversions, of which about 4,000 would have happened without any advertising (high-intent users buy anyway); the ads caused only about 3,636. Last click credits 7,299 conversions to the ads, twice the true incrementality. The shares matter more: social's true incremental share is 51.2 %, and last click gives it 21.0 %; search's true share is 11.0 %, and last click gives it 38.5 %. There are two reasons, independent of each other. First, search usually comes at the end of a path (people search once they have decided to buy), so the last-touch rule naturally favours it. Second, search and email mainly reach people who were going to buy anyway, so they "appear" on paths much more often than they "cause" conversions. First click and linear do little better, with mean absolute errors of 12.3 and 11.9 percentage points (last click 18.1).

Scope of the numbers: Which advertising channel should a sale be credited to? When may an A/B test stop? What is a customer still worth? This book presents mathematical models for teaching and research and is not marketing, investment or legal advice. Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's models and hypotheses; all data in the book are fictional or simulated.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 11.0, 11.9, 12.3, 18.1, 21.0, 3636

200,000 fictional users7,676 conversions (377 with no touch)
expected4,000 would happen without any ad, 3,636 are incremental
share of credit truth last click first click linear
Search11.0% 38.5% 6.4% 20.3%
Social51.2% 21.0% 31.2% 27.4%
Display22.0% 16.1% 46.7% 30.4%
Email15.8% 24.5% 15.7% 22.0%
mean absolute error of last click18.1%
mean absolute error of first click12.3%
mean absolute error of linear11.9%
last click credits 7,299 conversions to ads; the ads caused about 3,636

Key formula

\[E[Y \mid D=1] - E[Y \mid D=0] = \underbrace{E[Y(1) - Y(0) \mid D=1]}_{\text{曝光者的平均因果效应}} + \underbrace{E[Y(0) \mid D=1] - E[Y(0) \mid D=0]}_{\text{选择偏差}}.\]
Show the full program
# A FICTIONAL world of customer journeys whose true incrementality is known.
# 20 % of users have high purchase intent. Four channels: Search (S), Social (F),
# Display (D), Email (E). Exposure to S and E depends strongly on intent (selection);
# D is untargeted. Conversion probability = base(intent) + sum of the causal lifts
# of the channels seen (additive, so each channel's incremental effect is unique).
# Touch times: D and F early, S late, E anywhere; a conversion follows the last touch.
# Rule-based attribution (last, first, linear click) is compared with the truth.
import random

CH = "SFDE"
P_HIGH = 0.20
EXPO = {"S": (0.60, 0.10), "F": (0.35, 0.30), "D": (0.50, 0.50), "E": (0.60, 0.15)}
BASE = (0.08, 0.005)                                   # (high intent, low intent)
LIFT = {"S": 0.010, "F": 0.030, "D": 0.008, "E": 0.012}
WHEN = {"S": (0.6, 1.0), "F": (0.0, 0.7), "D": (0.0, 0.5), "E": (0.2, 1.0)}

def user(rng, sel=1.0):
    """One journey: (intent, channels in time order, converted)."""
    hi = rng.random() < P_HIGH
    touches = []
    for c in CH:
        p_hi, p_lo = EXPO[c]
        p = p_lo + sel * (p_hi - p_lo) if hi else p_lo   # sel = 0: no selection
        if rng.random() < p:
            touches.append((rng.uniform(*WHEN[c]), c))
    path = [c for _, c in sorted(touches)]
    p_conv = BASE[0 if hi else 1] + sum(LIFT[c] for c in path)
    return hi, path, rng.random() < p_conv

def truth(n, sel=1.0):
    """Expected incremental conversions per channel (exact, from the model)."""
    out = {}
    for c in CH:
        p_hi, p_lo = EXPO[c]
        expo = P_HIGH * (p_lo + sel * (p_hi - p_lo)) + (1 - P_HIGH) * p_lo
        out[c] = n * expo * LIFT[c]
    return out

def shares(d):
    tot = sum(d.values())
    return {c: d[c] / tot for c in CH}

def rules(data):
    last, first, linear = ({c: 0.0 for c in CH} for _ in range(3))
    for _, path, conv in data:
        if conv and path:
            last[path[-1]] += 1; first[path[0]] += 1
            for c in path:
                linear[c] += 1 / len(path)
    return last, first, linear

if __name__ == "__main__":
    n = 200_000
    rng = random.Random(2026)
    data = [user(rng) for _ in range(n)]
    conv = sum(d[2] for d in data)
    untouched = sum(d[2] for d in data if not d[1])
    tr = truth(n)
    base = n * (P_HIGH * BASE[0] + (1 - P_HIGH) * BASE[1])
    print(f"{n:,} fictional users: {conv:,} conversions ({untouched:,} with no touch)")
    print(f"expected: {base:,.0f} would happen without any ad,"
          f" {sum(tr.values()):,.0f} are incremental")
    last, first, linear = rules(data)
    print("share of credit       truth    last click   first click   linear")
    T, L, F, Li = shares(tr), shares(last), shares(first), shares(linear)
    names = {"S": "Search", "F": "Social", "D": "Display", "E": "Email"}
    for c in CH:
        print(f"   {names[c]:8s}         {T[c]:6.1%}     {L[c]:6.1%}       {F[c]:6.1%}"
              f"     {Li[c]:6.1%}")
    for lab, s in (("last click", L), ("first click", F), ("linear", Li)):
        mae = sum(abs(s[c] - T[c]) for c in CH) / len(CH)
        print(f"   mean absolute error of {lab:11s} {mae:6.1%}")
    credited = sum(last.values())
    print(f"last click credits {credited:,.0f} conversions to ads;"
          f" the ads caused about {sum(tr.values()):,.0f}")

Single source and citation

Fan, Y. (2026). Algorithmic Marketing, Multi-Touch Shapley Attribution and Stochastic Control of Acquisition Funnels (Version 1.0.0), Chapter 1: Attribution Is Not Incrementality: Last Click, Potential Outcomes and Randomised Holdouts. 5D Library. https://doi.org/10.5281/zenodo.23836264

DOI: 10.5281/zenodo.23836264 · Read the monograph · 5DL-B-048 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026048,
  author    = {Fan, Yafu},
  title     = {Algorithmic Marketing, Multi-Touch Shapley Attribution and Stochastic Control of Acquisition Funnels},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.23836264},
  url       = {https://doi.org/10.5281/zenodo.23836264}
}

Other chapter answers in this book