What does Synthetic Control for Site Redesigns conclude, in numbers, about “The Donor Pool: No Spillover, Enough Donors and Interpolation Bias”?

Answer: With three donors the mean pre-period error is 19.95, 0.745 of the panels fail the fit, and the lift estimate has a root mean squared error of 31.81, hardly better than before-after; the floor of the p-value is 0.2500, so significance is impossible however large the effect. With five donors the failure share is still 0.495 and the floor 0.1667. With ten it drops to 0.125, the lift error is 9.47 and the floor 0.0909, still short of five per cent. With twenty the pre-period error is 7.52, failures 0.030, the lift error 6.78 and the floor 0.0476, crossing the five per cent line for the first time. With thirty-three the pre-period error is 6.84, failures 0.000, the lift error 5.42 and the floor 0.0294.

Scope of the numbers: On an illustrative panel of thirty-four sites with a true lift of twelve orders a week, before-after reports 37.77, mostly because of the season; weighting thirty-three untouched donor sites with simplex-constrained weights into a synthetic site gives 8.83. Over two hundred panels the root mean squared error is 35.61 for before-after and 5.42 for synthetic control.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.000, 0.0294, 0.030, 0.0476, 0.0909, 0.125

JpreRMSPEfit>10.5lift RMSEmin p
319.950.74531.810.2500
512.470.49519.330.1667
108.660.1259.470.0909
207.520.0306.780.0476
336.840.0005.420.0294
Show the full program
# Chapter 5. p52: how many donors: fit quality versus the p-value floor.
# Panel as in p11 (illustrative, true lift +12). From the 33 untreated
# sites a random pool of J donors is drawn (fixed seed) for each of
# 200 panels; J = 3, 5, 10, 20, 33. Reported: mean pre RMSPE, share of
# panels whose pre RMSPE exceeds 1.5 times the site's noise sd (10.5),
# RMSE of the estimated lift, and the smallest attainable placebo
# p-value 1/(J + 1).
import math
import random
from operator import mul

# 示意面板:每站周订单 = 基准 + 季节载荷 * 季节 + 大盘载荷 * 大盘 + 噪声
T0, POST, NS = 52, 12, 34


def panel(rep=0, n=NS, t0=T0, post=POST, tau=12.0, start=10):
    rng = random.Random(20261113 + 7919 * rep)
    weeks = t0 + post
    mk = [0.0]
    for _ in range(weeks - 1):
        mk.append(mk[-1] + rng.gauss(0.05, 0.25))
    ss = [math.sin(2.0 * math.pi * (start + t) / 52.0)
          for t in range(weeks)]
    ys, cf, pars = [], [], []
    for i in range(n):
        if i == 0:
            mu, ra, rb, sd = 240.0, 0.20, 1.30, 7.0
        else:
            mu = rng.uniform(120.0, 360.0)
            ra = rng.uniform(0.04, 0.22)
            rb = rng.uniform(0.5, 1.7)
            sd = rng.uniform(5.0, 9.0)
        a, b = mu * ra, mu * rb / 20.0
        y = [mu + a * ss[t] + b * mk[t] + rng.gauss(0.0, sd)
             for t in range(weeks)]
        cf.append(y[:])
        if i == 0:
            for t in range(t0, weeks):
                y[t] = y[t] + tau
        ys.append(y)
        pars.append((mu, a, b, sd))
    return ys, cf, pars, ss, mk


def dot(a, b):
    return math.fsum(map(mul, a, b))


def mean(xs):
    return math.fsum(xs) / len(xs)


def rmspe(xs):
    return math.sqrt(math.fsum([x * x for x in xs]) / len(xs))


def gram(ys, t0):
    n = len(ys)
    pre = [y[:t0] for y in ys]
    g = [[0.0] * n for _ in range(n)]
    for i in range(n):
        for k in range(i, n):
            v = dot(pre[i], pre[k]) / t0
            g[i][k] = v
            g[k][i] = v
    return g


def solve(a, b):
    m = len(b)
    r = [row[:] + [b[i]] for i, row in enumerate(a)]
    for c in range(m):
        p = max(range(c, m), key=lambda i: abs(r[i][c]))
        r[c], r[p] = r[p], r[c]
        for i in range(c + 1, m):
            f = r[i][c] / r[c][c]
            if f:
                r[i] = [x - f * y for x, y in zip(r[i], r[c])]
    x = [0.0] * m
    for i in range(m - 1, -1, -1):
        s = math.fsum([r[i][k] * x[k] for k in range(i + 1, m)])
        x[i] = (r[i][m] - s) / r[i][i]
    return x


def eq_qp(g, c, sup):
    m = len(sup)
    a = [[2.0 * g[i][k] for k in sup] + [1.0] for i in sup]
    a.append([1.0] * m + [0.0])
    b = [2.0 * c[i] for i in sup] + [1.0]
    return solve(a, b)[:m]


def simplex_ls(g, c, tol=1e-10):
    # 精确有效集法:min w'Gw - 2c'w,w >= 0,权重和为 1
    j = len(c)
    k0 = min(range(j), key=lambda k: g[k][k] - 2.0 * c[k])
    w = [0.0] * j
    w[k0] = 1.0
    sup = [k0]
    for _ in range(4 * j + 20):
        gr = [2.0 * (math.fsum([g[i][k] * w[k] for k in sup]) - c[i])
              for i in range(j)]
        nu = -math.fsum([gr[k] for k in sup]) / len(sup)
        out = [k for k in range(j) if k not in sup]
        if not out:
            break
        kb = min(out, key=lambda k: gr[k] + nu)
        if gr[kb] + nu >= -tol:
            break
        sup = sorted(sup + [kb])
        while True:
            z = eq_qp(g, c, sup)
            if min(z) > 0.0:
                for k, v in zip(sup, z):
                    w[k] = v
                break
            step = min(w[k] / (w[k] - v) for k, v in zip(sup, z)
                       if v <= 0.0)
            for k, v in zip(sup, z):
                w[k] = w[k] + step * (v - w[k])
            sup = [k for k in sup if w[k] > 1e-12]
            for k in range(j):
                if k not in sup:
                    w[k] = 0.0
    return w


def synth(ys, g, tr, donors):
    gs = [[g[i][k] for k in donors] for i in donors]
    cs = [g[i][tr] for i in donors]
    w = simplex_ls(gs, cs)
    weeks = len(ys[tr])
    gap = [ys[tr][t] - dot(w, [ys[d][t] for d in donors])
           for t in range(weeks)]
    return w, gap


def ratio(gap, t0=T0):
    return rmspe(gap[t0:]) / rmspe(gap[:t0])


def placebo(ys, g, tr, pool, t0=T0):
    # 逐个捐赠站当处理站;处理站本身不进入安慰剂的捐赠池
    out = {}
    for j in pool:
        dn = [k for k in pool if k != j]
        out[j] = synth(ys, g, j, dn)[1]
    return out


if __name__ == "__main__":
    pick = random.Random(20261113)
    print("  J  preRMSPE  fit>10.5  lift RMSE  min p")
    for jn in (3, 5, 10, 20, 33):
        pre, err = [], []
        for r in range(200):
            ys, cf, pars, ss, mk = panel(r)
            pool = sorted(pick.sample(range(1, NS), jn))
            g = gram(ys, T0)
            w, gap = synth(ys, g, 0, pool)
            pre.append(rmspe(gap[:T0]))
            err.append(mean(gap[T0:]) - 12.0)
        bad = len([x for x in pre if x > 10.5]) / len(pre)
        print("{:3d}  {:8.2f}  {:8.3f}  {:9.2f}  {:.4f}".format(
            jn, mean(pre), bad, rmspe(err), 1 / (jn + 1)))

Single source and citation

Fan, Y. (2026). Synthetic Control and the Causal Effect of a Single-Site Redesign: Rebuilding the Counterfactual from Weighted Donor Sites and Placebo Permutation Tests (Version 1.0.0), Chapter 5: The Donor Pool: No Spillover, Enough Donors and Interpolation Bias. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-304 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026304,
  author    = {Fan, Yafu},
  title     = {Synthetic Control and the Causal Effect of a Single-Site Redesign: Rebuilding the Counterfactual from Weighted Donor Sites and Placebo Permutation Tests},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book