What does Synthetic Control for Site Redesigns conclude, in numbers, about “The Donor Pool: No Spillover, Enough Donors and Interpolation Bias”?
Answer: With three donors the mean pre-period error is 19.95, 0.745 of the panels fail the fit, and the lift estimate has a root mean squared error of 31.81, hardly better than before-after; the floor of the p-value is 0.2500, so significance is impossible however large the effect. With five donors the failure share is still 0.495 and the floor 0.1667. With ten it drops to 0.125, the lift error is 9.47 and the floor 0.0909, still short of five per cent. With twenty the pre-period error is 7.52, failures 0.030, the lift error 6.78 and the floor 0.0476, crossing the five per cent line for the first time. With thirty-three the pre-period error is 6.84, failures 0.000, the lift error 5.42 and the floor 0.0294.
Scope of the numbers: On an illustrative panel of thirty-four sites with a true lift of twelve orders a week, before-after reports 37.77, mostly because of the season; weighting thirty-three untouched donor sites with simplex-constrained weights into a synthetic site gives 8.83. Over two hundred panels the root mean squared error is 35.61 for before-after and 5.42 for synthetic control.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.000, 0.0294, 0.030, 0.0476, 0.0909, 0.125
| J | preRMSPE | fit>10.5 | lift RMSE | min p |
| 3 | 19.95 | 0.745 | 31.81 | 0.2500 |
| 5 | 12.47 | 0.495 | 19.33 | 0.1667 |
| 10 | 8.66 | 0.125 | 9.47 | 0.0909 |
| 20 | 7.52 | 0.030 | 6.78 | 0.0476 |
| 33 | 6.84 | 0.000 | 5.42 | 0.0294 |
Show the full program
# Chapter 5. p52: how many donors: fit quality versus the p-value floor.
# Panel as in p11 (illustrative, true lift +12). From the 33 untreated
# sites a random pool of J donors is drawn (fixed seed) for each of
# 200 panels; J = 3, 5, 10, 20, 33. Reported: mean pre RMSPE, share of
# panels whose pre RMSPE exceeds 1.5 times the site's noise sd (10.5),
# RMSE of the estimated lift, and the smallest attainable placebo
# p-value 1/(J + 1).
import math
import random
from operator import mul
# 示意面板:每站周订单 = 基准 + 季节载荷 * 季节 + 大盘载荷 * 大盘 + 噪声
T0, POST, NS = 52, 12, 34
def panel(rep=0, n=NS, t0=T0, post=POST, tau=12.0, start=10):
rng = random.Random(20261113 + 7919 * rep)
weeks = t0 + post
mk = [0.0]
for _ in range(weeks - 1):
mk.append(mk[-1] + rng.gauss(0.05, 0.25))
ss = [math.sin(2.0 * math.pi * (start + t) / 52.0)
for t in range(weeks)]
ys, cf, pars = [], [], []
for i in range(n):
if i == 0:
mu, ra, rb, sd = 240.0, 0.20, 1.30, 7.0
else:
mu = rng.uniform(120.0, 360.0)
ra = rng.uniform(0.04, 0.22)
rb = rng.uniform(0.5, 1.7)
sd = rng.uniform(5.0, 9.0)
a, b = mu * ra, mu * rb / 20.0
y = [mu + a * ss[t] + b * mk[t] + rng.gauss(0.0, sd)
for t in range(weeks)]
cf.append(y[:])
if i == 0:
for t in range(t0, weeks):
y[t] = y[t] + tau
ys.append(y)
pars.append((mu, a, b, sd))
return ys, cf, pars, ss, mk
def dot(a, b):
return math.fsum(map(mul, a, b))
def mean(xs):
return math.fsum(xs) / len(xs)
def rmspe(xs):
return math.sqrt(math.fsum([x * x for x in xs]) / len(xs))
def gram(ys, t0):
n = len(ys)
pre = [y[:t0] for y in ys]
g = [[0.0] * n for _ in range(n)]
for i in range(n):
for k in range(i, n):
v = dot(pre[i], pre[k]) / t0
g[i][k] = v
g[k][i] = v
return g
def solve(a, b):
m = len(b)
r = [row[:] + [b[i]] for i, row in enumerate(a)]
for c in range(m):
p = max(range(c, m), key=lambda i: abs(r[i][c]))
r[c], r[p] = r[p], r[c]
for i in range(c + 1, m):
f = r[i][c] / r[c][c]
if f:
r[i] = [x - f * y for x, y in zip(r[i], r[c])]
x = [0.0] * m
for i in range(m - 1, -1, -1):
s = math.fsum([r[i][k] * x[k] for k in range(i + 1, m)])
x[i] = (r[i][m] - s) / r[i][i]
return x
def eq_qp(g, c, sup):
m = len(sup)
a = [[2.0 * g[i][k] for k in sup] + [1.0] for i in sup]
a.append([1.0] * m + [0.0])
b = [2.0 * c[i] for i in sup] + [1.0]
return solve(a, b)[:m]
def simplex_ls(g, c, tol=1e-10):
# 精确有效集法:min w'Gw - 2c'w,w >= 0,权重和为 1
j = len(c)
k0 = min(range(j), key=lambda k: g[k][k] - 2.0 * c[k])
w = [0.0] * j
w[k0] = 1.0
sup = [k0]
for _ in range(4 * j + 20):
gr = [2.0 * (math.fsum([g[i][k] * w[k] for k in sup]) - c[i])
for i in range(j)]
nu = -math.fsum([gr[k] for k in sup]) / len(sup)
out = [k for k in range(j) if k not in sup]
if not out:
break
kb = min(out, key=lambda k: gr[k] + nu)
if gr[kb] + nu >= -tol:
break
sup = sorted(sup + [kb])
while True:
z = eq_qp(g, c, sup)
if min(z) > 0.0:
for k, v in zip(sup, z):
w[k] = v
break
step = min(w[k] / (w[k] - v) for k, v in zip(sup, z)
if v <= 0.0)
for k, v in zip(sup, z):
w[k] = w[k] + step * (v - w[k])
sup = [k for k in sup if w[k] > 1e-12]
for k in range(j):
if k not in sup:
w[k] = 0.0
return w
def synth(ys, g, tr, donors):
gs = [[g[i][k] for k in donors] for i in donors]
cs = [g[i][tr] for i in donors]
w = simplex_ls(gs, cs)
weeks = len(ys[tr])
gap = [ys[tr][t] - dot(w, [ys[d][t] for d in donors])
for t in range(weeks)]
return w, gap
def ratio(gap, t0=T0):
return rmspe(gap[t0:]) / rmspe(gap[:t0])
def placebo(ys, g, tr, pool, t0=T0):
# 逐个捐赠站当处理站;处理站本身不进入安慰剂的捐赠池
out = {}
for j in pool:
dn = [k for k in pool if k != j]
out[j] = synth(ys, g, j, dn)[1]
return out
if __name__ == "__main__":
pick = random.Random(20261113)
print(" J preRMSPE fit>10.5 lift RMSE min p")
for jn in (3, 5, 10, 20, 33):
pre, err = [], []
for r in range(200):
ys, cf, pars, ss, mk = panel(r)
pool = sorted(pick.sample(range(1, NS), jn))
g = gram(ys, T0)
w, gap = synth(ys, g, 0, pool)
pre.append(rmspe(gap[:T0]))
err.append(mean(gap[T0:]) - 12.0)
bad = len([x for x in pre if x > 10.5]) / len(pre)
print("{:3d} {:8.2f} {:8.3f} {:9.2f} {:.4f}".format(
jn, mean(pre), bad, rmspe(err), 1 / (jn + 1)))Single source and citation
Fan, Y. (2026). Synthetic Control and the Causal Effect of a Single-Site Redesign: Rebuilding the Counterfactual from Weighted Donor Sites and Placebo Permutation Tests (Version 1.0.0), Chapter 5: The Donor Pool: No Spillover, Enough Donors and Interpolation Bias. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-304 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026304,
author = {Fan, Yafu},
title = {Synthetic Control and the Causal Effect of a Single-Site Redesign: Rebuilding the Counterfactual from Weighted Donor Sites and Placebo Permutation Tests},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Three Confounders in a Before-After Comparison: How Season, Market and Noise Pose as a Redesign Effect
- Chapter 2 · Simplex-Constrained Least Squares: How Donor Weights Are Found and Why They May Not Be Negative
- Chapter 3 · Pre-Period Fit Is the Entry Ticket: Root Mean Squared Prediction Error, the Convex Hull and Hold-Out Checks
- Chapter 4 · The Placebo Permutation Test: the Post/Pre Ratio, Ranks and the Floor of the p-Value
- Chapter 6 · Power and the Length of the Evaluation: How Many Weeks to See Five Per Cent
- Chapter 7 · Eastern and Western Sources: "Take by Kind, Give by Kind" in the Mozi and Mill's Method of Difference
- Chapter 8 · Closing the Loop: Evaluating Single-Site Redesigns across Thirty-Four Sites, with a Pre-Registered Protocol