What does Simpson's Paradox and Back-Door Adjustment conclude, in numbers, about “Forced Stratification and Adjustment of Multi-Branch Reports: the Closed Loop, the Tool and a Pre-Registered Test”?
Answer: Chapter 8 sets out the closed loop of forced stratification and adjustment for multi-branch reports, the distilled Stratified Reversal Detector and a pre-registered randomised holdout: when the branch is the only confounder the adjusted estimate is falsified in 0.033 of replications, and with a hidden confounder in 0.637. The book contains 23 runnable Python programs.
Scope of the numbers: Branches, plans and conversions are illustrative; all calculations are generated by the book's programs. This is general analysis, not individualised business advice.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.033, 0.637
true average effect +0.0412; 300 replications world I (branch only): mean |error| 0.0227 falsified 0.033 world II (hidden trait): mean |error| 0.0720 falsified 0.637
Show the full program
# Chapter 8. p83: the falsification test, a randomised holdout.
# History: 1200 customers, plan chosen by branch (as in p12) and, in
# world II, also by a hidden trait U inside branches. Holdout: 300
# per plan per branch, randomised. The pre-registered rule: the
# standardised historical gap is falsified when it differs from the
# holdout gap by more than 1.96 combined standard errors.
# World I: branch is the only confounder; world II: U confounds too.
# Fixed seed; parameters illustrative.
import math
import random
PZ = [0.37, 0.31, 0.32]
PA = [0.12, 0.57, 0.83]
BASE = [0.70, 0.36, 0.10]
EFF = [0.06, 0.02, 0.04]
TRUE = math.fsum(PZ[s] * EFF[s] for s in range(3))
REPS = 300
def buy(rng, z, a, u):
return rng.random() < BASE[z] + (EFF[z] if a else 0) + 0.15 * u
def history(rng, hidden):
c = [[[0, 0], [0, 0]] for _ in range(3)]
for _ in range(1200):
r = rng.random()
z = 0 if r < PZ[0] else (1 if r < PZ[0] + PZ[1] else 2)
u = 1 if rng.random() < 0.5 else 0
p = PA[z] + (0.15 if u else -0.15) * hidden
a = 1 if rng.random() < p else 0
c[z][a][0] = c[z][a][0] + 1
c[z][a][1] = c[z][a][1] + (1 if buy(rng, z, a, u) else 0)
return c
def holdout(rng):
c = [[[300, 0], [300, 0]] for _ in range(3)]
for z in range(3):
for a in (0, 1):
for _ in range(300):
u = 1 if rng.random() < 0.5 else 0
c[z][a][1] = c[z][a][1] + (1 if buy(rng, z, a, u) else 0)
return c
def standardised(c):
n = math.fsum(c[z][a][0] for z in range(3) for a in (0, 1))
d, v = [], []
for z in range(3):
(n0, y0), (n1, y1) = c[z][0], c[z][1]
p0, p1 = y0 / n0, y1 / n1
w = (n0 + n1) / n
d.append(w * (p1 - p0))
v.append(w * w * (p1 * (1 - p1) / n1 + p0 * (1 - p0) / n0))
return math.fsum(d), math.fsum(v)
def run(rng, hidden):
err, fals = [], 0
for _ in range(REPS):
h, hv = standardised(history(rng, hidden))
o, ov = standardised(holdout(rng))
err.append(abs(h - TRUE))
if abs(h - o) > 1.96 * math.sqrt(hv + ov):
fals = fals + 1
return math.fsum(err) / REPS, fals / REPS
def main():
rng = random.Random(20261113)
print("true average effect {:+.4f}; {} replications".format(
TRUE, REPS))
for hidden, name in ((0, "world I (branch only)"),
(1, "world II (hidden trait)")):
e, f = run(rng, hidden)
print("{}: mean |error| {:.4f} falsified {:.3f}".format(
name, e, f))
if __name__ == "__main__":
main()Single source and citation
Fan, Y. (2026). Simpson's Paradox and the Back-Door Criterion: Reversals When Branch Data Are Pooled and De-Confounding with Causal Graphs (Version 1.0.0), Chapter 8: Forced Stratification and Adjustment of Multi-Branch Reports: the Closed Loop, the Tool and a Pre-Registered Test. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-302 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026302,
author = {Fan, Yafu},
title = {Simpson's Paradox and the Back-Door Criterion: Reversals When Branch Data Are Pooled and De-Confounding with Causal Graphs},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Every Branch Wins, the Total Loses: the Reversal Condition for 2 x 2 x k Tables
- Chapter 2 · How Common Are Reversals? The Smallest Example, Random Tables and Sampling Noise
- Chapter 3 · The Back-Door Adjustment Formula: Standardisation, Inverse Probability Weighting and a Structural Model with a Known Answer
- Chapter 4 · One Table, Three Causal Graphs: Confounder, Mediator and Collider
- Chapter 5 · Enumerating Back-Door Sets: d-Separation, M-Bias and Counting Causal Graphs
- Chapter 6 · Sampling Error in Stratified Estimates: Intervals, Power and Sparse Strata
- Chapter 7 · Classics East and West: the Lüshi Chunqiu on Examining Hearsay, and Yule 1903