What does Resampling for Shop Experiments conclude, in numbers, about “Eastern and Western Sources: Tasting One Slice of Meat in the Lüshi Chunqiu and Fisher's Lady Tasting Tea”?
Answer: With d equal to zero, the simulated rejection rates of the four designs are 0.0521, 0.0151, 0.0408 and 0.0358, matching the exact values 0.0500, 0.0143, 0.0400 and 0.0352 in the first row, which shows the simulation is sound. With d equal to one, the eight-cup design passes her only 0.1412 of the time and the twelve-cup design 0.3595; with d equal to two, 0.4934 and 0.8313; with d equal to three, 0.8349 and 0.9878.
Scope of the numbers: Parameters are illustrative; all calculations are generated by the book's programs; the book is general business analysis, not individualised business advice. Marked boxes separate published evidence, runnable Python calculations (20 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.0143, 0.0151, 0.0352, 0.0358, 0.0400, 0.0408
| P(reject H0 of no ability), 20000 trials per cell | ||||
| d | 6 cups | 8 cups | 12 cups | 8 singly |
| size | 0.0500 | 0.0143 | 0.0400 | 0.0352 |
| 0.0 | 0.0521 | 0.0151 | 0.0408 | 0.0358 |
| 1.0 | 0.2599 | 0.1412 | 0.3595 | 0.2366 |
| 1.5 | 0.4368 | 0.2945 | 0.6190 | 0.4291 |
| 2.0 | 0.6214 | 0.4934 | 0.8313 | 0.6301 |
| 3.0 | 0.8904 | 0.8349 | 0.9878 | 0.9050 |
Show the full program
# Chapter 7. p72: how much ability does Fisher's design detect?
# Model (signal detection, illustrative): each milk-first cup gives the
# taster a sensation score N(d, 1), each tea-first cup N(0, 1); d is
# her discrimination ability. Fixed-split designs: she names the m
# highest-scoring cups as milk-first. Singly judged design: each of 8
# cups (fair coin for its kind) is called milk-first if its score
# exceeds d / 2. Rejection rules and exact sizes from p21: 6 cups all
# right (1/20), 8 cups all right (1/70), 12 cups >= 5 of 6 (37/924),
# 8 singly judged >= 7 right (9/256). 20000 seeded trials per cell.
import random
def fixed_split(m, need, d, reps, rng):
hit = 0
for _ in range(reps):
cups = ([(rng.gauss(d, 1.0), 1) for _ in range(m)]
+ [(rng.gauss(0.0, 1.0), 0) for _ in range(m)])
cups.sort(reverse=True)
right = len([c for c in cups[:m] if c[1] == 1])
hit += right >= need
return hit / reps
def singly(d, reps, rng):
hit = 0
for _ in range(reps):
right = 0
for _ in range(8):
milk = rng.random() < 0.5
score = rng.gauss(d if milk else 0.0, 1.0)
right += (score > d / 2) == milk
hit += right >= 7
return hit / reps
if __name__ == "__main__":
rng = random.Random(20261125)
reps = 20000
print("P(reject H0 of no ability), {} trials per cell".format(reps))
print(" d 6 cups 8 cups 12 cups 8 singly")
print(" size 0.0500 0.0143 0.0400 0.0352")
for d in (0.0, 1.0, 1.5, 2.0, 3.0):
print(" {:4.1f} {:.4f} {:.4f} {:.4f} {:.4f}".format(
d, fixed_split(3, 3, d, reps, rng),
fixed_split(4, 4, d, reps, rng),
fixed_split(6, 5, d, reps, rng), singly(d, reps, rng)))Single source and citation
Fan, Y. (2026). The Bootstrap and Permutation Tests: Resampling Inference for Small Shop Experiments without Normality Assumptions (Version 1.0.0), Chapter 7: Eastern and Western Sources: Tasting One Slice of Meat in the Lüshi Chunqiu and Fisher's Lady Tasting Tea. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-308 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026308,
author = {Fan, Yafu},
title = {The Bootstrap and Permutation Tests: Resampling Inference for Small Shop Experiments without Normality Assumptions},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Nominal Five Per Cent, Actual False Alarms: the Type I Error of the t Test on Small Skewed Samples
- Chapter 2 · From Eight Teacups to Twelve Trading Days: Why the Exact Permutation Test Does Not Over-Report
- Chapter 3 · Splits Too Many to List: Seeded Monte Carlo Permutation and the Add-One Rule
- Chapter 4 · Bootstrap Intervals: Percentile, Bias-Corrected and Accelerated, and Undercoverage in Small Samples
- Chapter 5 · When Exchangeability Fails: Unequal Variances, Before-and-After Comparisons and Correlated Days Break the Permutation Test Too
- Chapter 6 · Designing a Shop Experiment: No Peeking, No Early Stopping, Pair First, Size by Simulation
- Chapter 8 · An Analysis Template for Small Shop Experiments, the Permutation Tester and a Plan to Refute the Book