What does Four-Player Imperfect Information conclude, in numbers, about “Practice and a Pre-Registered Test”?
Answer: Chapter 8 gives a four-step checklist and a pre-registered duplicate-design policy comparison scored at the first differing decision: smallest effect of interest 0.010 buy-ins per scored decision, 12,000 scored decisions, the decision rule written as code; the conclusion holds only against the fixed opponent pool used. Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's models and hypotheses; apart from cited literature data, all data in the book are synthetic.
Scope of the numbers: Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's models and hypotheses; apart from cited literature data, all data in the book are synthetic.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.010, 12000
Duplicate design: the same deals replayed under two policies, scored at the first differing decision, against a fixed opponent pool. SESOI 0.010 buy-ins per scored decision; paired s.d. 0.120. normal theory: n = 1,513.1 -> 1,514 scored decisions generator's actual s.d.: 0.1256 (the planning value was 0.120) normal theory at the generator's s.d.: 1,657 scored decisions simulation with the heavy-tailed paired difference: n = 1,657: power 0.880 (s.e. 0.013) n = 12,000: power 1.000 (s.e. 0.000) n = 15,000: power 1.000 (s.e. 0.000) n = 18,000: power 1.000 (s.e. 0.000) smallest n with power - 2 s.e. >= 0.90: 12,000 one-sided error rate with no effect: 0.033 (nominal 0.025) planned: 12,000 scored decisions. At roughly one scored decision per hand and 60 hands per session, that is about 200 sessions of play, or a few weeks of continuous self-play for a software agent. sensitivity (normal theory): SESOI \ paired s.d. 0.08 0.12 0.20 0.005 buy-ins 2,690 6,053 16,812 0.010 buy-ins 673 1,514 4,203 0.020 buy-ins 169 379 1,051 Two things this sizing makes explicit. First, a human cannot run this trial: 12,000 scored decisions is beyond what anyone plays deliberately, so the comparison belongs to software agents replaying stored deals, and the conclusions then apply to the policies rather than to people. Second, the SESOI of 0.010 buy-ins per decision is small in absolute terms and still needs this many observations, which is the quantitative reason the strategy debates in this game remain unsettled: the sample sizes required have
Show the full program
# Chapter 8. Sample size for the pre-registered policy comparison. Design: a
# duplicate design in which the SAME deals are replayed with two decision
# policies, scored at the first decision at which the policies differ, against
# a fixed opponent pool. Primary
# outcome: the mean points difference per scored decision. The pairing is what
# makes the trial feasible (p11); the program sizes it on the paired standard
# deviation and then checks the normal-theory answer against a simulation with
# a realistic, heavy-tailed outcome distribution, because mahjong scores are
# not normal and the t test's behaviour in the tail matters.
import math
import random
from statistics import NormalDist
Z = NormalDist().inv_cdf
SESOI = 0.010 # buy-ins per scored decision
SD_DIFF = 0.120 # paired s.d. from the duplicate design (p11)
def n_paired(d=SESOI, sd=SD_DIFF, power=0.90, alpha=0.05):
return (Z(1 - alpha / 2) + Z(power)) ** 2 * sd ** 2 / d ** 2
# a heavy-tailed paired difference: mostly zero, occasionally a large swing
def draw_diff(rng, delta):
u = rng.random()
if u < 0.70:
return delta + rng.gauss(0.0, 0.02) # small or no consequence
if u < 0.94:
return delta + rng.gauss(0.0, 0.12) # a normal-sized swing
# a big hand changes hands
return delta + rng.gauss(0.0, 0.45)
def one_trial(rng, n, delta):
xs = [draw_diff(rng, delta) for _ in range(n)]
m = math.fsum(xs) / n
v = math.fsum((x - m) ** 2 for x in xs) / (n - 1)
se = math.sqrt(v / n)
t = m / se if se > 0 else 0.0
return t > Z(0.975), m
if __name__ == "__main__":
print(
"Duplicate design: the same deals replayed under two policies,"
" scored at"
)
print("the first differing decision, against a fixed opponent pool.")
print("SESOI {:.3f} buy-ins per scored decision; paired s.d. {:.3f}.\n"
.format(SESOI, SD_DIFF))
n0 = n_paired()
print("normal theory: n = {:,.1f} -> {:,} scored decisions"
.format(n0, math.ceil(n0)))
rng = random.Random(5514)
# empirical s.d. of the heavy-tailed generator, for comparison
samp = [draw_diff(rng, 0.0) for _ in range(200000)]
m = math.fsum(samp) / len(samp)
sd_emp = math.sqrt(
math.fsum((x - m) ** 2 for x in samp) / (len(samp) - 1)
)
print("generator's actual s.d.: {:.4f} (the planning value was {:.3f})"
.format(sd_emp, SD_DIFF))
n_adj = n_paired(sd=sd_emp)
print("normal theory at the generator's s.d.: {:,} scored decisions"
.format(math.ceil(n_adj)))
print("\nsimulation with the heavy-tailed paired difference:")
chosen = None
for n in (math.ceil(n_adj), 12000, 15000, 18000):
res = [one_trial(rng, n, SESOI) for _ in range(600)]
pw = sum(r[0] for r in res) / len(res)
se = math.sqrt(pw * (1 - pw) / len(res))
print(" n = {:6,}: power {:.3f} (s.e. {:.3f})".format(n, pw, se))
if chosen is None and pw - 2 * se >= 0.90:
chosen = n
if chosen is None:
chosen = 18000
t1 = sum(one_trial(rng, chosen, 0.0)[0] for _ in range(600)) / 600
print(" smallest n with power - 2 s.e. >= 0.90: {:,}".format(chosen))
print(
" one-sided error rate with no effect: {:.3f} (nominal 0.025)"
.format(t1)
)
print(
"\nplanned: {:,} scored decisions. At roughly one scored decision per"
" hand".format(chosen)
)
print(
"and 60 hands per session, that is about {:,.0f} sessions of play, or"
.format(chosen / 60)
)
print("a few weeks of continuous self-play for a software agent.")
print("\nsensitivity (normal theory):")
print(" SESOI \\ paired s.d. 0.08 0.12 0.20")
for d in (0.005, 0.010, 0.020):
print(" {:.3f} buy-ins ".format(d) + "".join(
"{:10,.0f}".format(math.ceil(n_paired(d=d, sd=s)))
for s in (0.08, 0.12, 0.20)))
print(
"\nTwo things this sizing makes explicit. First, a human cannot run"
" this"
)
print(
"trial: {:,} scored decisions is beyond what anyone plays"
" deliberately, so".format(chosen)
)
print(
"the comparison belongs to software agents replaying stored deals,"
" and the"
)
print(
"conclusions then apply to the policies rather than to people."
" Second, the"
)
print(
"SESOI of {:.3f} buy-ins per decision is small in absolute terms and"
" still".format(SESOI)
)
print(
"needs this many observations, which is the quantitative reason the"
" strategy"
)
print(
"debates in this game remain unsettled: the sample sizes required"
" have"
)
print("rarely been assembled.")Single source and citation
Fan, Y. (2026). Imperfect-Information Stochastic Games, Bayesian Hand-Reading and Counterfactual Regret Minimisation in Four-Player Tile Dynamics (Version 1.0.0), Chapter 8: Practice and a Pre-Registered Test. 5D Library. https://doi.org/10.5281/zenodo.24020464
DOI: 10.5281/zenodo.24020464 · Read the monograph · 5DL-B-055 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026055,
author = {Fan, Yafu},
title = {Imperfect-Information Stochastic Games, Bayesian Hand-Reading and Counterfactual Regret Minimisation in Four-Player Tile Dynamics},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.24020464},
url = {https://doi.org/10.5281/zenodo.24020464}
}Other chapter answers in this book
- Chapter 1 · Scale: Why the Information Sets Cannot Be Listed
- Chapter 2 · The Tile Pool: Hypergeometric Inference and a Result That Runs Backwards
- Chapter 3 · Hand Reading: Suji Is a Deduction, Not a Regularity
- Chapter 4 · Shanten, Acceptance and the Race: Ignoring Three Players Overstates the Win
- Chapter 5 · Folding: Optimal Stopping, and a Conclusion Stated Backwards
- Chapter 6 · Solving the Game, and the Money
- Chapter 7 · Classics Read Side by Side: The Substantial and the Empty, the Orthodox and the Unorthodox, and the Fog of War