What does Regression Discontinuity at Membership Tiers conclude, in numbers, about “Closing the Loop: Evaluating a Membership-Tier Threshold, the Distilled Tool and a Pre-Registered Plan”?
Answer: Chapter 8 gives a complete procedure for evaluating a membership-tier threshold, the distilled Threshold Discontinuity Effect Estimator and a pre-registered test. The book contains 23 runnable Python programs.
Scope of the numbers: Parameters are illustrative; all calculations are generated by the book's programs. This is general analysis, not individualised business advice, and the book neither recommends nor supplies misleading threshold copy designed to provoke top-ups.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python only
| pilot se at n = 20000: 0.2451 | ||
| effect m | customers needed | within h of the line |
| 0.5 | 37722 | 12992 |
| 1.0 | 9431 | 3249 |
| 1.5 | 4192 | 1444 |
| 2.5 | 1509 | 520 |
| simulated power at n = 9431 for m = 1: 0.78 |
Show the full program
# Chapter 8. p81: how many customers a threshold study needs.
# The local linear RD standard error shrinks like 1 / sqrt(n). From one
# pilot sample of 20000 customers (seed 20261113, line c = 60, h = 8)
# we take se0 and compute the sample needed to detect a local effect of
# size m with a two-sided 5% test and 80% power:
# n(m) = 20000 * ((1.959964 + 0.841621) * se0 / m)^2.
# A Monte Carlo check (100 samples, seed 20261181) sets the benefit
# so that the local effect equals m = 1 and measures the power at n(1).
# Illustrative design; a real pilot must use the shop's own data.
import math
import random
C, H, N0 = 60.0, 8.0, 20000
ZA, ZB = 1.959964, 0.841621
def draw(rng, n, base=2.0, c=C):
rows = []
for _ in range(n):
u = rng.gauss(0.0, 1.0)
x = 50.0 + 12.0 * u + 8.0 * rng.gauss(0.0, 1.0)
d = 1 if x >= c else 0
g = 0.15 * (x - c) + 0.0003 * (x - c) ** 3
y = 20.0 + 6.0 * u + g + (base + u) * d
rows.append((x, y + rng.gauss(0.0, 3.0), u))
return rows
def wls(pts, c, h):
xs, ys, ws = [], [], []
for x, y in pts:
t = abs(x - c) / h
if t < 1.0:
xs.append(x - c)
ys.append(y)
ws.append(1.0 - t)
s0 = math.fsum(ws)
s1 = math.fsum(w * x for w, x in zip(ws, xs))
s2 = math.fsum(w * x * x for w, x in zip(ws, xs))
t0 = math.fsum(w * y for w, y in zip(ws, ys))
t1 = math.fsum(w * x * y for w, x, y in zip(ws, xs, ys))
det = s0 * s2 - s1 * s1
a = (s2 * t0 - s1 * t1) / det
b = (s0 * t1 - s1 * t0) / det
va = math.fsum((w * (s2 - s1 * x) / det * (y - a - b * x)) ** 2
for w, x, y in zip(ws, xs, ys))
return a, va
def rd(rows, c, h):
al, vl = wls([(r[0], r[1]) for r in rows if r[0] < c], c, h)
ar, vr = wls([(r[0], r[1]) for r in rows if r[0] >= c], c, h)
return ar - al, math.sqrt(vl + vr)
if __name__ == "__main__":
_, se0 = rd(draw(random.Random(20261113), N0), C, H)
print("pilot se at n = {}: {:.4f}".format(N0, se0))
print("effect m customers needed within h of the line")
share = 6888 / N0
need = {}
for m in (0.5, 1.0, 1.5, 2.5):
n = int(math.ceil(N0 * ((ZA + ZB) * se0 / m) ** 2))
need[m] = n
print("{:8.1f} {:16d} {:20d}".format(
m, n, int(math.ceil(n * share))))
rng = random.Random(20261181)
base = 1.0 - 12.0 / 208.0 * (C - 50.0)
hit = 0
for _ in range(100):
est, se = rd(draw(rng, need[1.0], base), C, H)
if abs(est / se) > ZA:
hit = hit + 1
print("simulated power at n = {} for m = 1: {:.2f}".format(
need[1.0], hit / 100))Single source and citation
Fan, Y. (2026). Regression-Discontinuity Designs at Membership-Tier Thresholds: Local Causal Identification on Either Side of a Spend-Based Upgrade Line (Version 1.0.0), Chapter 8: Closing the Loop: Evaluating a Membership-Tier Threshold, the Distilled Tool and a Pre-Registered Plan. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-303 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026303,
author = {Fan, Yafu},
title = {Regression-Discontinuity Designs at Membership-Tier Thresholds: Local Causal Identification on Either Side of a Spend-Based Upgrade Line},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Either Side of the VIP Line: Selection Bias, the Naive Gap and a Local Comparison at the Threshold
- Chapter 2 · Local Linear Regression: Triangular-Kernel Weighted Least Squares, Boundary Bias and an Effect That Holds Only at the Threshold
- Chapter 3 · Choosing the Bandwidth: Trading Bias for Variance, a Monte Carlo Optimum and Boundary Cross-Validation
- Chapter 4 · The McCrary Density Test: Local Lines on Either Side of a Histogram, Power and Blind Spots
- Chapter 5 · Topping Up to Upgrade: How Precisely Manipulable Spending Breaks the Design
- Chapter 6 · Three Robustness Checks: High-Order Polynomials, Placebo Thresholds and the Fuzzy Design
- Chapter 7 · Sources East and West: Rank for Count in the Book of Lord Shang and the Scholarship Discontinuity of 1960