What does Simpson's Paradox and Back-Door Adjustment conclude, in numbers, about “Forced Stratification and Adjustment of Multi-Branch Reports: the Closed Loop, the Tool and a Pre-Registered Test”?

Answer: Chapter 8 sets out the closed loop of forced stratification and adjustment for multi-branch reports, the distilled Stratified Reversal Detector and a pre-registered randomised holdout: when the branch is the only confounder the adjusted estimate is falsified in 0.033 of replications, and with a hidden confounder in 0.637. The book contains 23 runnable Python programs.

Scope of the numbers: Branches, plans and conversions are illustrative; all calculations are generated by the book's programs. This is general analysis, not individualised business advice.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.033, 0.637

true average effect +0.0412; 300 replications
world I  (branch only): mean |error| 0.0227  falsified 0.033
world II (hidden trait): mean |error| 0.0720  falsified 0.637
Show the full program
# Chapter 8. p83: the falsification test, a randomised holdout.
# History: 1200 customers, plan chosen by branch (as in p12) and, in
# world II, also by a hidden trait U inside branches. Holdout: 300
# per plan per branch, randomised. The pre-registered rule: the
# standardised historical gap is falsified when it differs from the
# holdout gap by more than 1.96 combined standard errors.
# World I: branch is the only confounder; world II: U confounds too.
# Fixed seed; parameters illustrative.
import math
import random

PZ = [0.37, 0.31, 0.32]
PA = [0.12, 0.57, 0.83]
BASE = [0.70, 0.36, 0.10]
EFF = [0.06, 0.02, 0.04]
TRUE = math.fsum(PZ[s] * EFF[s] for s in range(3))
REPS = 300


def buy(rng, z, a, u):
    return rng.random() < BASE[z] + (EFF[z] if a else 0) + 0.15 * u


def history(rng, hidden):
    c = [[[0, 0], [0, 0]] for _ in range(3)]
    for _ in range(1200):
        r = rng.random()
        z = 0 if r < PZ[0] else (1 if r < PZ[0] + PZ[1] else 2)
        u = 1 if rng.random() < 0.5 else 0
        p = PA[z] + (0.15 if u else -0.15) * hidden
        a = 1 if rng.random() < p else 0
        c[z][a][0] = c[z][a][0] + 1
        c[z][a][1] = c[z][a][1] + (1 if buy(rng, z, a, u) else 0)
    return c


def holdout(rng):
    c = [[[300, 0], [300, 0]] for _ in range(3)]
    for z in range(3):
        for a in (0, 1):
            for _ in range(300):
                u = 1 if rng.random() < 0.5 else 0
                c[z][a][1] = c[z][a][1] + (1 if buy(rng, z, a, u) else 0)
    return c


def standardised(c):
    n = math.fsum(c[z][a][0] for z in range(3) for a in (0, 1))
    d, v = [], []
    for z in range(3):
        (n0, y0), (n1, y1) = c[z][0], c[z][1]
        p0, p1 = y0 / n0, y1 / n1
        w = (n0 + n1) / n
        d.append(w * (p1 - p0))
        v.append(w * w * (p1 * (1 - p1) / n1 + p0 * (1 - p0) / n0))
    return math.fsum(d), math.fsum(v)


def run(rng, hidden):
    err, fals = [], 0
    for _ in range(REPS):
        h, hv = standardised(history(rng, hidden))
        o, ov = standardised(holdout(rng))
        err.append(abs(h - TRUE))
        if abs(h - o) > 1.96 * math.sqrt(hv + ov):
            fals = fals + 1
    return math.fsum(err) / REPS, fals / REPS


def main():
    rng = random.Random(20261113)
    print("true average effect {:+.4f}; {} replications".format(
        TRUE, REPS))
    for hidden, name in ((0, "world I  (branch only)"),
                         (1, "world II (hidden trait)")):
        e, f = run(rng, hidden)
        print("{}: mean |error| {:.4f}  falsified {:.3f}".format(
            name, e, f))


if __name__ == "__main__":
    main()

Single source and citation

Fan, Y. (2026). Simpson's Paradox and the Back-Door Criterion: Reversals When Branch Data Are Pooled and De-Confounding with Causal Graphs (Version 1.0.0), Chapter 8: Forced Stratification and Adjustment of Multi-Branch Reports: the Closed Loop, the Tool and a Pre-Registered Test. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-302 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026302,
  author    = {Fan, Yafu},
  title     = {Simpson's Paradox and the Back-Door Criterion: Reversals When Branch Data Are Pooled and De-Confounding with Causal Graphs},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book