What does Regression Discontinuity at Membership Tiers conclude, in numbers, about “Three Robustness Checks: High-Order Polynomials, Placebo Thresholds and the Fuzzy Design”?
Answer: If the benefits really act at the line of 60, then at other points where no rule changes, say 48 or 72, next year's spending should not jump. Program p62 places fake thresholds at 36, 42, 48 and 54 below the line and at 66, 72 and 78 above it, each using only customers on the same side (non-VIPs for fake thresholds below the line, VIPs for those above) so that the real jump cannot leak in, with a bandwidth of 6. It then repeats the seven tests on 20 fresh samples and counts the false alarms.
Scope of the numbers: Parameters are illustrative; all calculations are generated by the book's programs. This is general analysis, not individualised business advice, and the book neither recommends nor supplies misleading threshold copy designed to provoke top-ups.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 20, 36, 42, 48, 54, 60
| line | sample | estimate | se | z |
|---|---|---|---|---|
| 36 | non-VIP | 0.090 | 0.305 | 0.29 |
| 42 | non-VIP | -0.106 | 0.265 | -0.40 |
| 48 | non-VIP | 0.924 | 0.247 | 3.74 |
| 54 | non-VIP | 0.145 | 0.249 | 0.58 |
| 60 | all | 2.830 | 0.281 | 10.06 |
| 66 | VIP | -0.119 | 0.369 | -0.32 |
| 72 | VIP | 0.274 | 0.472 | 0.58 |
| 78 | VIP | -0.775 | 0.698 | -1.11 |
| placebo lines with |z| > 1.96: 1 of 7 | ||||
| 20 more samples: 4 of 140 placebo tests with |z| > 1.96 | ||||
| largest placebo |z| in those samples: 2.77 |
Show the full program
# Chapter 6. p62: placebo lines where nothing happens.
# Data as in p11 (seed 20261113, 20000 customers, real line c = 60).
# At fake lines we run the same local linear RD (triangular kernel,
# h = 6), using only customers on one side of the real line so that the
# real jump cannot leak in: fake lines below 60 use non-VIPs only, fake
# lines above 60 use VIPs only. A sound design shows |z| < 1.96 at most
# placebo lines and a clear jump only at the real one. A second part
# repeats the seven placebo tests on 20 fresh samples (seeds 20261200
# to 20261219) to show the false alarm rate. Illustrative.
import math
import random
C, H = 60.0, 6.0
def draw(rng, n, sel=1.0, c=C):
rows = []
for _ in range(n):
u = rng.gauss(0.0, 1.0)
x = 50.0 + 12.0 * u + 8.0 * rng.gauss(0.0, 1.0)
d = 1 if x >= c else 0
g = 0.15 * (x - c) + 0.0003 * (x - c) ** 3
y = 20.0 + 6.0 * sel * u + g + (2.0 + u) * d
rows.append((x, y + rng.gauss(0.0, 3.0), u))
return rows
def wls(pts, c, h):
xs, ys, ws = [], [], []
for x, y in pts:
t = abs(x - c) / h
if t < 1.0:
xs.append(x - c)
ys.append(y)
ws.append(1.0 - t)
s0 = math.fsum(ws)
s1 = math.fsum(w * x for w, x in zip(ws, xs))
s2 = math.fsum(w * x * x for w, x in zip(ws, xs))
t0 = math.fsum(w * y for w, y in zip(ws, ys))
t1 = math.fsum(w * x * y for w, x, y in zip(ws, xs, ys))
det = s0 * s2 - s1 * s1
a = (s2 * t0 - s1 * t1) / det
b = (s0 * t1 - s1 * t0) / det
va = math.fsum((w * (s2 - s1 * x) / det * (y - a - b * x)) ** 2
for w, x, y in zip(ws, xs, ys))
return a, va
def jump(pts, c, h):
al, vl = wls([p for p in pts if p[0] < c], c, h)
ar, vr = wls([p for p in pts if p[0] >= c], c, h)
return ar - al, math.sqrt(vl + vr)
if __name__ == "__main__":
rows = draw(random.Random(20261113), 20000)
below = [(r[0], r[1]) for r in rows if r[0] < C]
above = [(r[0], r[1]) for r in rows if r[0] >= C]
allp = below + above
print("line sample estimate se z")
hits = 0
for c in (36.0, 42.0, 48.0, 54.0, 60.0, 66.0, 72.0, 78.0):
pts = allp if c == C else (below if c < C else above)
name = "all" if c == C else ("non-VIP" if c < C else "VIP")
est, se = jump(pts, c, H)
if c != C and abs(est / se) > 1.96:
hits = hits + 1
print("{:4.0f} {:8s} {:9.3f} {:7.3f} {:6.2f}".format(
c, name, est, se, est / se))
print("placebo lines with |z| > 1.96: {} of 7".format(hits))
alarms, zmax = 0, 0.0
for k in range(20):
rows = draw(random.Random(20261200 + k), 20000)
below = [(r[0], r[1]) for r in rows if r[0] < C]
above = [(r[0], r[1]) for r in rows if r[0] >= C]
for c in (36.0, 42.0, 48.0, 54.0, 66.0, 72.0, 78.0):
est, se = jump(below if c < C else above, c, H)
z = abs(est / se)
zmax = max(zmax, z)
if z > 1.96:
alarms = alarms + 1
print("20 more samples: {} of 140 placebo tests with |z| > 1.96".format(
alarms))
print("largest placebo |z| in those samples: {:.2f}".format(zmax))Single source and citation
Fan, Y. (2026). Regression-Discontinuity Designs at Membership-Tier Thresholds: Local Causal Identification on Either Side of a Spend-Based Upgrade Line (Version 1.0.0), Chapter 6: Three Robustness Checks: High-Order Polynomials, Placebo Thresholds and the Fuzzy Design. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-303 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026303,
author = {Fan, Yafu},
title = {Regression-Discontinuity Designs at Membership-Tier Thresholds: Local Causal Identification on Either Side of a Spend-Based Upgrade Line},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Either Side of the VIP Line: Selection Bias, the Naive Gap and a Local Comparison at the Threshold
- Chapter 2 · Local Linear Regression: Triangular-Kernel Weighted Least Squares, Boundary Bias and an Effect That Holds Only at the Threshold
- Chapter 3 · Choosing the Bandwidth: Trading Bias for Variance, a Monte Carlo Optimum and Boundary Cross-Validation
- Chapter 4 · The McCrary Density Test: Local Lines on Either Side of a Histogram, Power and Blind Spots
- Chapter 5 · Topping Up to Upgrade: How Precisely Manipulable Spending Breaks the Design
- Chapter 7 · Sources East and West: Rank for Count in the Book of Lord Shang and the Scholarship Discontinuity of 1960
- Chapter 8 · Closing the Loop: Evaluating a Membership-Tier Threshold, the Distilled Tool and a Pre-Registered Plan