What does Resampling for Shop Experiments conclude, in numbers, about “Bootstrap Intervals: Percentile, Bias-Corrected and Accelerated, and Undercoverage in Small Samples”?

Answer: Chapter 4 gives percentile and simplified BCa bootstrap intervals; with five exponential observations a nominal ninety-five per cent percentile interval covers 0.798 of the time.

Scope of the numbers: Parameters are illustrative; all calculations are generated by the book's programs; the book is general business analysis, not individualised business advice. Marked boxes separate published evidence, runnable Python calculations (20 programs), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.798

coverage of nominal 0.95 intervals, 500 data sets
populationntpctBCa| pct all below mu
exp(1)50.8920.7980.808 |0.178
exp(1)100.9040.8580.876 |0.122
exp(1)200.9320.9180.932 |0.066
logn(0,1)50.7940.7060.732 |0.278
logn(0,1)100.8460.8000.830 |0.184
logn(0,1)200.8880.8820.888 |0.116
Show the full program
# Chapter 4. p42: do bootstrap intervals cover at small n?
# Model: draw n days from a skewed population with known mean
# (exponential(1), mean 1; lognormal(0, 1), mean exp(0.5)); build the
# nominal 95 percent t, percentile and simplified BCa intervals of
# p41 (B = 500 resamples each) and record whether the true mean is
# inside; the last column is the share of percentile intervals lying
# wholly below the true mean. R = 500 seeded data sets per
# cell, so a coverage near 0.95 carries a Monte Carlo standard error
# of about 0.01. Sizes and populations are illustrative.
import math
import random
from statistics import NormalDist

def betacf(a, b, x):
    tiny = 1e-300
    qab, qap, qam = a + b, a + 1.0, a - 1.0
    c = 1.0
    d = 1.0 - qab * x / qap
    d = 1.0 / (d if abs(d) > tiny else tiny)
    h = d
    for m in range(1, 300):
        m2 = 2 * m
        aa = m * (b - m) * x / ((qam + m2) * (a + m2))
        d = 1.0 + aa * d
        d = 1.0 / (d if abs(d) > tiny else tiny)
        c = 1.0 + aa / c
        c = c if abs(c) > tiny else tiny
        h *= d * c
        aa = -(a + m) * (qab + m) * x / ((a + m2) * (qap + m2))
        d = 1.0 + aa * d
        d = 1.0 / (d if abs(d) > tiny else tiny)
        c = 1.0 + aa / c
        c = c if abs(c) > tiny else tiny
        de = d * c
        h *= de
        if abs(de - 1.0) < 1e-15:
            break
    return h


def betai(a, b, x):
    if x <= 0.0:
        return 0.0
    if x >= 1.0:
        return 1.0
    lb = (math.lgamma(a + b) - math.lgamma(a) - math.lgamma(b)
          + a * math.log(x) + b * math.log(1.0 - x))
    bt = math.exp(lb)
    if x < (a + 1.0) / (a + b + 2.0):
        return bt * betacf(a, b, x) / a
    return 1.0 - bt * betacf(b, a, 1.0 - x) / b


def t_two_tail(t, v):
    return betai(v / 2.0, 0.5, v / (v + t * t))


def t_cdf(t, v):
    half = 0.5 * t_two_tail(t, v)
    return 1.0 - half if t > 0 else half


def t_crit(alpha, v):
    lo, hi = 0.0, 1000.0
    for _ in range(200):
        mid = 0.5 * (lo + hi)
        if t_two_tail(mid, v) > alpha:
            lo = mid
        else:
            hi = mid
    return 0.5 * (lo + hi)

N01 = NormalDist()


def mean(v):
    return math.fsum(v) / len(v)


def quantile(sv, q):
    pos = q * (len(sv) - 1)
    i = int(pos)
    if i >= len(sv) - 1:
        return sv[-1]
    return sv[i] + (pos - i) * (sv[i + 1] - sv[i])


def intervals(x, nboot, rng, tc):
    n = len(x)
    m = mean(x)
    s = math.sqrt(math.fsum((v - m) ** 2 for v in x) / (n - 1))
    out = [(m - tc * s / math.sqrt(n), m + tc * s / math.sqrt(n))]
    bm = sorted(mean(rng.choices(x, k=n)) for _ in range(nboot))
    out.append((quantile(bm, 0.025), quantile(bm, 0.975)))
    below = len([v for v in bm if v < m])
    z0 = N01.inv_cdf(min(max(below / nboot, 1e-6), 1 - 1e-6))
    jack = [mean(x[:i] + x[i + 1:]) for i in range(n)]
    jm = mean(jack)
    d = [jm - v for v in jack]
    den = math.fsum(v ** 2 for v in d) ** 1.5
    a = math.fsum(v ** 3 for v in d) / (6.0 * den) if den > 0 else 0.0
    ends = []
    for q in (0.025, 0.975):
        zq = N01.inv_cdf(q)
        ends.append(quantile(bm, N01.cdf(z0 + (z0 + zq)
                                         / (1 - a * (z0 + zq)))))
    out.append(tuple(ends))
    return out


if __name__ == "__main__":
    rng = random.Random(20261119)
    reps, nboot = 500, 500
    print("coverage of nominal 0.95 intervals, {} data sets".format(reps))
    print("  population   n    t      pct    BCa   | pct all below mu")
    for name, draw, mu in (
            ("exp(1)", lambda r: r.expovariate(1.0), 1.0),
            ("logn(0,1)", lambda r: r.lognormvariate(0.0, 1.0),
             math.exp(0.5))):
        for n in (5, 10, 20):
            tc = t_crit(0.05, n - 1)
            cov = [0, 0, 0]
            miss_hi = 0
            for _ in range(reps):
                x = [draw(rng) for _ in range(n)]
                ivs = intervals(x, nboot, rng, tc)
                for j, (lo, hi) in enumerate(ivs):
                    cov[j] += lo <= mu <= hi
                miss_hi += ivs[1][1] < mu
            print("  {:10s} {:3d}  {:.3f}  {:.3f}  {:.3f} |   {:.3f}".format(
                name, n, cov[0] / reps, cov[1] / reps, cov[2] / reps,
                miss_hi / reps))

Single source and citation

Fan, Y. (2026). The Bootstrap and Permutation Tests: Resampling Inference for Small Shop Experiments without Normality Assumptions (Version 1.0.0), Chapter 4: Bootstrap Intervals: Percentile, Bias-Corrected and Accelerated, and Undercoverage in Small Samples. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-308 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026308,
  author    = {Fan, Yafu},
  title     = {The Bootstrap and Permutation Tests: Resampling Inference for Small Shop Experiments without Normality Assumptions},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book