What does Regression Discontinuity at Membership Tiers conclude, in numbers, about “Three Robustness Checks: High-Order Polynomials, Placebo Thresholds and the Fuzzy Design”?

Answer: If the benefits really act at the line of 60, then at other points where no rule changes, say 48 or 72, next year's spending should not jump. Program p62 places fake thresholds at 36, 42, 48 and 54 below the line and at 66, 72 and 78 above it, each using only customers on the same side (non-VIPs for fake thresholds below the line, VIPs for those above) so that the real jump cannot leak in, with a bandwidth of 6. It then repeats the seven tests on 20 fresh samples and counts the false alarms.

Scope of the numbers: Parameters are illustrative; all calculations are generated by the book's programs. This is general analysis, not individualised business advice, and the book neither recommends nor supplies misleading threshold copy designed to provoke top-ups.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 20, 36, 42, 48, 54, 60

linesampleestimatesez
36non-VIP0.0900.3050.29
42non-VIP-0.1060.265-0.40
48non-VIP0.9240.2473.74
54non-VIP0.1450.2490.58
60all2.8300.28110.06
66VIP-0.1190.369-0.32
72VIP0.2740.4720.58
78VIP-0.7750.698-1.11
placebo lines with |z| > 1.96: 1 of 7
20 more samples: 4 of 140 placebo tests with |z| > 1.96
largest placebo |z| in those samples: 2.77
Show the full program
# Chapter 6. p62: placebo lines where nothing happens.
# Data as in p11 (seed 20261113, 20000 customers, real line c = 60).
# At fake lines we run the same local linear RD (triangular kernel,
# h = 6), using only customers on one side of the real line so that the
# real jump cannot leak in: fake lines below 60 use non-VIPs only, fake
# lines above 60 use VIPs only. A sound design shows |z| < 1.96 at most
# placebo lines and a clear jump only at the real one. A second part
# repeats the seven placebo tests on 20 fresh samples (seeds 20261200
# to 20261219) to show the false alarm rate. Illustrative.
import math
import random

C, H = 60.0, 6.0


def draw(rng, n, sel=1.0, c=C):
    rows = []
    for _ in range(n):
        u = rng.gauss(0.0, 1.0)
        x = 50.0 + 12.0 * u + 8.0 * rng.gauss(0.0, 1.0)
        d = 1 if x >= c else 0
        g = 0.15 * (x - c) + 0.0003 * (x - c) ** 3
        y = 20.0 + 6.0 * sel * u + g + (2.0 + u) * d
        rows.append((x, y + rng.gauss(0.0, 3.0), u))
    return rows


def wls(pts, c, h):
    xs, ys, ws = [], [], []
    for x, y in pts:
        t = abs(x - c) / h
        if t < 1.0:
            xs.append(x - c)
            ys.append(y)
            ws.append(1.0 - t)
    s0 = math.fsum(ws)
    s1 = math.fsum(w * x for w, x in zip(ws, xs))
    s2 = math.fsum(w * x * x for w, x in zip(ws, xs))
    t0 = math.fsum(w * y for w, y in zip(ws, ys))
    t1 = math.fsum(w * x * y for w, x, y in zip(ws, xs, ys))
    det = s0 * s2 - s1 * s1
    a = (s2 * t0 - s1 * t1) / det
    b = (s0 * t1 - s1 * t0) / det
    va = math.fsum((w * (s2 - s1 * x) / det * (y - a - b * x)) ** 2
                   for w, x, y in zip(ws, xs, ys))
    return a, va


def jump(pts, c, h):
    al, vl = wls([p for p in pts if p[0] < c], c, h)
    ar, vr = wls([p for p in pts if p[0] >= c], c, h)
    return ar - al, math.sqrt(vl + vr)


if __name__ == "__main__":
    rows = draw(random.Random(20261113), 20000)
    below = [(r[0], r[1]) for r in rows if r[0] < C]
    above = [(r[0], r[1]) for r in rows if r[0] >= C]
    allp = below + above
    print("line   sample     estimate      se       z")
    hits = 0
    for c in (36.0, 42.0, 48.0, 54.0, 60.0, 66.0, 72.0, 78.0):
        pts = allp if c == C else (below if c < C else above)
        name = "all" if c == C else ("non-VIP" if c < C else "VIP")
        est, se = jump(pts, c, H)
        if c != C and abs(est / se) > 1.96:
            hits = hits + 1
        print("{:4.0f}   {:8s} {:9.3f}  {:7.3f}  {:6.2f}".format(
            c, name, est, se, est / se))
    print("placebo lines with |z| > 1.96: {} of 7".format(hits))
    alarms, zmax = 0, 0.0
    for k in range(20):
        rows = draw(random.Random(20261200 + k), 20000)
        below = [(r[0], r[1]) for r in rows if r[0] < C]
        above = [(r[0], r[1]) for r in rows if r[0] >= C]
        for c in (36.0, 42.0, 48.0, 54.0, 66.0, 72.0, 78.0):
            est, se = jump(below if c < C else above, c, H)
            z = abs(est / se)
            zmax = max(zmax, z)
            if z > 1.96:
                alarms = alarms + 1
    print("20 more samples: {} of 140 placebo tests with |z| > 1.96".format(
        alarms))
    print("largest placebo |z| in those samples: {:.2f}".format(zmax))

Single source and citation

Fan, Y. (2026). Regression-Discontinuity Designs at Membership-Tier Thresholds: Local Causal Identification on Either Side of a Spend-Based Upgrade Line (Version 1.0.0), Chapter 6: Three Robustness Checks: High-Order Polynomials, Placebo Thresholds and the Fuzzy Design. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-303 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026303,
  author    = {Fan, Yafu},
  title     = {Regression-Discontinuity Designs at Membership-Tier Thresholds: Local Causal Identification on Either Side of a Spend-Based Upgrade Line},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book