What does Simpson's Paradox and Back-Door Adjustment conclude, in numbers, about “Sampling Error in Stratified Estimates: Intervals, Power and Sparse Strata”?

Answer: Detecting an effect of 0.05 with 80 % power needs 393.5 customers per plan per branch, rounded up to 394, or 2364 customers across three branches and two plans. The table shows how power grows with sample size: 0.292 at 100 customers per plan per branch, 0.515 at 200, 0.806 at 400, 0.933 at 600, 0.979 at 800 and 0.998 at 1200. The simulated power at 400 is 0.814, within simulation error of the closed-form 0.806.

Scope of the numbers: Branches, plans and conversions are illustrative; all calculations are generated by the book's programs. This is general analysis, not individualised business advice.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.292, 0.515, 0.806, 0.814, 0.933, 0.979

n per plan per branch for 80 % power393.5 -> 394
customers in total2364
n se power
100 0.03540.292
200 0.02500.515
400 0.01770.806
600 0.01450.933
800 0.01250.979
1200 0.01020.998
simulated power at n400 (1000 runs): 0.814
Show the full program
# Chapter 6. p62: power of the branch-standardised gap.
# Rates from p12 (A beats B by 0.05 in each branch), branch weights
# w = (0.4, 0.3, 0.3); n customers per plan per branch. Normal
# approximation: power = Phi(d / se - z), se^2 = sum_s w_s^2 V_s / n,
# V_s = pA qA + pB qB. A seeded simulation checks one row.
import math
import random
from statistics import NormalDist

RA = [0.80, 0.45, 0.20]
RB = [0.75, 0.40, 0.15]
W = [0.4, 0.3, 0.3]
D = 0.05
ND = NormalDist()
ZA = ND.inv_cdf(0.975)


def se(n):
    v = [RA[s] * (1 - RA[s]) + RB[s] * (1 - RB[s]) for s in range(3)]
    return math.sqrt(math.fsum(W[s] ** 2 * v[s] for s in range(3)) / n)


def power(n):
    return ND.cdf(D / se(n) - ZA)


def simulate(n, reps, rng):
    hit = 0
    for _ in range(reps):
        d, v = [], []
        for s in range(3):
            a = len([1 for _ in range(n) if rng.random() < RA[s]]) / n
            b = len([1 for _ in range(n) if rng.random() < RB[s]]) / n
            d.append(W[s] * (a - b))
            v.append(W[s] ** 2 * (a * (1 - a) + b * (1 - b)) / n)
        if math.fsum(d) / math.sqrt(math.fsum(v)) > ZA:
            hit = hit + 1
    return hit / reps


def main():
    zb = ND.inv_cdf(0.8)
    v = [RA[s] * (1 - RA[s]) + RB[s] * (1 - RB[s]) for s in range(3)]
    need = (ZA + zb) ** 2 * math.fsum(
        W[s] ** 2 * v[s] for s in range(3)) / D ** 2
    print("n per plan per branch for 80 % power: {:.1f} -> {}".format(
        need, math.ceil(need)))
    print("customers in total: {}".format(6 * math.ceil(need)))
    print("   n   se      power")
    for n in (100, 200, 400, 600, 800, 1200):
        print("{:5d} {:.4f} {:8.3f}".format(n, se(n), power(n)))
    rng = random.Random(20261113)
    print("simulated power at n = 400 (1000 runs): {:.3f}".format(
        simulate(400, 1000, rng)))


if __name__ == "__main__":
    main()

Single source and citation

Fan, Y. (2026). Simpson's Paradox and the Back-Door Criterion: Reversals When Branch Data Are Pooled and De-Confounding with Causal Graphs (Version 1.0.0), Chapter 6: Sampling Error in Stratified Estimates: Intervals, Power and Sparse Strata. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-302 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026302,
  author    = {Fan, Yafu},
  title     = {Simpson's Paradox and the Back-Door Criterion: Reversals When Branch Data Are Pooled and De-Confounding with Causal Graphs},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book