What does Synthetic Control for Site Redesigns conclude, in numbers, about “Closing the Loop: Evaluating Single-Site Redesigns across Thirty-Four Sites, with a Pre-Registered Protocol”?

Answer: Chapter 8 gives a thirty-four-site evaluation sheet, the distilled Synthetic Control Evaluator and a pre-registered protocol: for a lift of five per cent of volume the single-site test has power of about 0.780 at thirteen weeks; several sites can be pooled by the sum of their ranks, and five sites that each rank third give a combined p of 0.000066. Marked boxes separate published evidence, runnable Python calculations (23 programs), classical sources and the author's hypotheses.

Scope of the numbers: On an illustrative panel of thirty-four sites with a true lift of twelve orders a week, before-after reports 37.77, mostly because of the season; weighting thirty-three untouched donor sites with simplex-constrained weights into a synthetic site gives 8.83. Over two hundred panels the root mean squared error is 35.61 for before-after and 5.42 for synthetic control.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.780

power to reach p = 1/34, 150 panels
lift4wk8wk13wk17wk26wkshortest n for 0.80
60.2130.2470.2530.2670.233none
90.2930.4330.5330.5000.500none
120.5200.6270.7800.7930.753none
150.7270.8730.9270.9400.8678
Show the full program
# Chapter 8. p82: pre-registered post length for a target power.
# Panel as in p11 (illustrative) with 26 post weeks and true lift 0;
# 150 panels. As in p61 the placebo ratios do not depend on the lift,
# so for each post length n the power to reach p = 1/34 is computed
# for lifts of 6, 9, 12 and 15 orders/week. The pre-registered rule:
# choose the shortest n (from 4, 8, 13, 17, 26) with power >= 0.80;
# "none" if no listed n reaches it.
import math
import random
from operator import mul

# 示意面板:每站周订单 = 基准 + 季节载荷 * 季节 + 大盘载荷 * 大盘 + 噪声
T0, POST, NS = 52, 12, 34


def panel(rep=0, n=NS, t0=T0, post=POST, tau=12.0, start=10):
    rng = random.Random(20261113 + 7919 * rep)
    weeks = t0 + post
    mk = [0.0]
    for _ in range(weeks - 1):
        mk.append(mk[-1] + rng.gauss(0.05, 0.25))
    ss = [math.sin(2.0 * math.pi * (start + t) / 52.0)
          for t in range(weeks)]
    ys, cf, pars = [], [], []
    for i in range(n):
        if i == 0:
            mu, ra, rb, sd = 240.0, 0.20, 1.30, 7.0
        else:
            mu = rng.uniform(120.0, 360.0)
            ra = rng.uniform(0.04, 0.22)
            rb = rng.uniform(0.5, 1.7)
            sd = rng.uniform(5.0, 9.0)
        a, b = mu * ra, mu * rb / 20.0
        y = [mu + a * ss[t] + b * mk[t] + rng.gauss(0.0, sd)
             for t in range(weeks)]
        cf.append(y[:])
        if i == 0:
            for t in range(t0, weeks):
                y[t] = y[t] + tau
        ys.append(y)
        pars.append((mu, a, b, sd))
    return ys, cf, pars, ss, mk


def dot(a, b):
    return math.fsum(map(mul, a, b))


def mean(xs):
    return math.fsum(xs) / len(xs)


def rmspe(xs):
    return math.sqrt(math.fsum([x * x for x in xs]) / len(xs))


def gram(ys, t0):
    n = len(ys)
    pre = [y[:t0] for y in ys]
    g = [[0.0] * n for _ in range(n)]
    for i in range(n):
        for k in range(i, n):
            v = dot(pre[i], pre[k]) / t0
            g[i][k] = v
            g[k][i] = v
    return g


def solve(a, b):
    m = len(b)
    r = [row[:] + [b[i]] for i, row in enumerate(a)]
    for c in range(m):
        p = max(range(c, m), key=lambda i: abs(r[i][c]))
        r[c], r[p] = r[p], r[c]
        for i in range(c + 1, m):
            f = r[i][c] / r[c][c]
            if f:
                r[i] = [x - f * y for x, y in zip(r[i], r[c])]
    x = [0.0] * m
    for i in range(m - 1, -1, -1):
        s = math.fsum([r[i][k] * x[k] for k in range(i + 1, m)])
        x[i] = (r[i][m] - s) / r[i][i]
    return x


def eq_qp(g, c, sup):
    m = len(sup)
    a = [[2.0 * g[i][k] for k in sup] + [1.0] for i in sup]
    a.append([1.0] * m + [0.0])
    b = [2.0 * c[i] for i in sup] + [1.0]
    return solve(a, b)[:m]


def simplex_ls(g, c, tol=1e-10):
    # 精确有效集法:min w'Gw - 2c'w,w >= 0,权重和为 1
    j = len(c)
    k0 = min(range(j), key=lambda k: g[k][k] - 2.0 * c[k])
    w = [0.0] * j
    w[k0] = 1.0
    sup = [k0]
    for _ in range(4 * j + 20):
        gr = [2.0 * (math.fsum([g[i][k] * w[k] for k in sup]) - c[i])
              for i in range(j)]
        nu = -math.fsum([gr[k] for k in sup]) / len(sup)
        out = [k for k in range(j) if k not in sup]
        if not out:
            break
        kb = min(out, key=lambda k: gr[k] + nu)
        if gr[kb] + nu >= -tol:
            break
        sup = sorted(sup + [kb])
        while True:
            z = eq_qp(g, c, sup)
            if min(z) > 0.0:
                for k, v in zip(sup, z):
                    w[k] = v
                break
            step = min(w[k] / (w[k] - v) for k, v in zip(sup, z)
                       if v <= 0.0)
            for k, v in zip(sup, z):
                w[k] = w[k] + step * (v - w[k])
            sup = [k for k in sup if w[k] > 1e-12]
            for k in range(j):
                if k not in sup:
                    w[k] = 0.0
    return w


def synth(ys, g, tr, donors):
    gs = [[g[i][k] for k in donors] for i in donors]
    cs = [g[i][tr] for i in donors]
    w = simplex_ls(gs, cs)
    weeks = len(ys[tr])
    gap = [ys[tr][t] - dot(w, [ys[d][t] for d in donors])
           for t in range(weeks)]
    return w, gap


def ratio(gap, t0=T0):
    return rmspe(gap[t0:]) / rmspe(gap[:t0])


def placebo(ys, g, tr, pool, t0=T0):
    # 逐个捐赠站当处理站;处理站本身不进入安慰剂的捐赠池
    out = {}
    for j in pool:
        dn = [k for k in pool if k != j]
        out[j] = synth(ys, g, j, dn)[1]
    return out


if __name__ == "__main__":
    pool = list(range(1, NS))
    lifts = (6.0, 9.0, 12.0, 15.0)
    wins = (4, 8, 13, 17, 26)
    hit = {(d, n): 0 for d in lifts for n in wins}
    reps = 150
    for r in range(reps):
        ys, cf, pars, ss, mk = panel(r, post=26, tau=0.0)
        g = gram(ys, T0)
        gaps = placebo(ys, g, 0, pool)
        g0 = synth(ys, g, 0, pool)[1]
        pre0 = rmspe(g0[:T0])
        for n in wins:
            top = max(rmspe(gp[T0:T0 + n]) / rmspe(gp[:T0])
                      for gp in gaps.values())
            for d in lifts:
                st = rmspe([x + d for x in g0[T0:T0 + n]]) / pre0
                if st > top:
                    hit[(d, n)] = hit[(d, n)] + 1
    print("power to reach p = 1/34, {} panels".format(reps))
    print("lift  " + "  ".join("{:2d}wk ".format(n) for n in wins)
          + "  shortest n for 0.80")
    for d in lifts:
        pw = [hit[(d, n)] / reps for n in wins]
        ok = [n for n, p in zip(wins, pw) if p >= 0.80]
        print("{:4.0f}  ".format(d) + "  ".join(
            "{:.3f}".format(p) for p in pw) + "  {}".format(
                ok[0] if ok else "none"))

Single source and citation

Fan, Y. (2026). Synthetic Control and the Causal Effect of a Single-Site Redesign: Rebuilding the Counterfactual from Weighted Donor Sites and Placebo Permutation Tests (Version 1.0.0), Chapter 8: Closing the Loop: Evaluating Single-Site Redesigns across Thirty-Four Sites, with a Pre-Registered Protocol. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-304 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026304,
  author    = {Fan, Yafu},
  title     = {Synthetic Control and the Causal Effect of a Single-Site Redesign: Rebuilding the Counterfactual from Weighted Donor Sites and Placebo Permutation Tests},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book