5D Library › Imperfect-Information Stochastic Games, Bayesian Hand-Reading and Counterfactual Regret Minimisation in Four-Player Tile Dynamics · 中文
Chapter 8. Practice and a Pre-Registered Test
Note
This chapter is a research protocol and a method of reading, not a gambling guide. The trial in Chapter 8 is a comparison of strategies between software agents and involves no wagering of real money. The checklist helps a reader put the right questions to any claim about how to play. The last section of the chapter uses a calculation to show why staking money on an unmeasured edge is a quantifiable error.
Part one: a four-step checklist
The first seven chapters condense into a four-step checklist for reading any claim of the form “this way of playing is better”.
- Which class of quantity. Does the claim rest on quantities that look only at one’s own tiles (shanten, acceptance, hand value), or on quantities that depend on the opponents’ tiles (deal-in probability, what an opponent waits on)? The first can be computed exactly; the second can only be estimated under a model and a prior (Chapter 1).
- Direction. Is any link in the argument pointed the opposite way from the mathematics it rests on? Counting acceptance tiles overstates wide waits early (Chapter 2); ignoring the other three players overstates the win probability by 1.36 to 1.79 and biases toward slow, valuable hands (Chapter 4, p05); folding protects only against dealing in, so the self-draw term cancels (Chapter 5).
- Who the opponent is. Does the claim assume an adaptive or a non-adaptive opponent? Hand reading is a best response to a non-adaptive opponent (Chapter 3); the value of mixing is conditional on the opponent’s volume of observation (Chapter 6). Has “better against some opponent pool” been stated as “better”?
- Sample. How many hands support the claim? By program p11, an improvement of 0.01 buy-ins a hand needs about 39,766 hands per arm to be detected (Chapter 6 and this chapter). Has personal experience been treated as evidence?
Applying the checklist to the usual strategic reasoning about this game gives the following.
| Step | How the usual reasoning fares | What can be supported |
|---|---|---|
| Which class of quantity | The two classes are mixed, and estimates of the second are often treated as computations of the first | Conclusions about one’s own tiles are comparatively reliable; conclusions about opponents must be quoted together with their prior |
| Direction | At least three links point the wrong way (counting acceptance early, ignoring competing risk, “when in doubt, fold”) | These need rewriting, not supplementing |
| Who the opponent is | The opponent pool is usually not stated | Conclusions hold only for the class of opponent assumed |
| Sample | Mainly personal experience and impression | Confident opinions about marginal decisions are not supported by the number of hands |
Part two: how many hands are needed
Before designing any comparison, work out how large it has to be.
Calculation
# Chapter 8. How many hands does it take to know whether a change helped? This
# is the single most useful number in the book, and it is uncomfortable. With
# a per-hand outcome standard deviation of s and a true improvement of d per
# hand, distinguishing d from zero at the usual error rates needs
# n = 2 (z_{1-a/2} + z_{1-b})^2 s^2 / d^2 (two independent samples)
# n = (z_{1-a/2} + z_{1-b})^2 s^2 / d^2 (one sample against a known
# mean)
# and because s is large relative to any plausible d, n is in the thousands to
# hundreds of thousands. The program also computes the paired/variance-reduced
# version, which is the only thing that makes the measurement feasible at all.
import math
from statistics import NormalDist
Z = NormalDist().inv_cdf
# per-hand outcome s.d. in buy-ins, from the p09 distribution
SD_HAND = 0.435
def n_two_sample(d, sd=SD_HAND, power=0.90, alpha=0.05):
return 2 * (Z(1 - alpha / 2) + Z(power)) ** 2 * sd ** 2 / d ** 2
def n_paired(d, sd_diff, power=0.90, alpha=0.05):
return (Z(1 - alpha / 2) + Z(power)) ** 2 * sd_diff ** 2 / d ** 2
if __name__ == "__main__":
print("Per-hand outcome s.d. {:.3f} buy-ins.\n".format(SD_HAND))
print("A. Hands needed per arm to detect an improvement of d per hand,")
print("two independent samples, two-sided 0.05, 90 % power:")
print(
" d per hand as a share of the s.d. hands per arm hands in"
" total"
)
for d in (0.100, 0.050, 0.020, 0.010, 0.005, 0.002):
n = n_two_sample(d)
print(" {:10.3f} {:23.3f} {:15,.0f} {:16,.0f}"
.format(d, d / SD_HAND, math.ceil(n), 2 * math.ceil(n)))
print(
"\n A 0.01 improvement per hand is a large edge in this game, and"
" detecting"
)
print(
" it takes about {:,.0f} hands per arm. At 60 hands in a session"
" that is".format(math.ceil(n_two_sample(0.01)))
)
print(
" about {:,.0f} sessions - which is why individual experience is"
" almost".format(math.ceil(n_two_sample(0.01)) / 60)
)
print(
" worthless as evidence about strategy, and why the confident"
" opinions"
)
print(
" that players hold about marginal decisions are not supportable"
" by the"
)
print(" number of hands they have played.")
print(
"\nB. What a player can actually learn from a season. With {:,}"
" hands:".format(10000)
)
n = 10000
print(" the smallest improvement detectable at 90 % power is")
d_min = math.sqrt(2 * (Z(0.975) + Z(0.90)) ** 2 * SD_HAND ** 2 / n)
print(" {:.4f} buy-ins per hand ({:.1%} of the outcome s.d.)."
.format(d_min, d_min / SD_HAND))
print(
" Anything smaller is invisible at that sample size, however real."
)
print(
"\nC. The variance reduction that makes it feasible. If the same"
" hands are"
)
print(
"replayed under both policies - the duplicate-game design of"
" Chapter 8 -"
)
print(
"the deal is held fixed and only the decisions differ, so the"
" comparison"
)
print("is paired and the relevant s.d. is that of the DIFFERENCE:")
print(
" s.d. of the difference as a share of the raw s.d. hands for d"
" = 0.01"
)
for sd_diff in (0.42, 0.30, 0.20, 0.12, 0.08):
print(" {:22.3f} {:28.1%} {:19,.0f}"
.format(sd_diff, sd_diff / SD_HAND,
math.ceil(n_paired(0.01, sd_diff))))
print(
"\n If duplicating the deals cuts the standard deviation of the"
" difference"
)
print(
" to {:.2f}, the same effect needs {:,.0f} hands instead of"
" {:,.0f}, a".format(
0.12,
math.ceil(n_paired(0.01, 0.12)),
2 * math.ceil(n_two_sample(0.01)),
)
)
print(
" reduction of about {:.0f}-fold. This is the whole reason"
" Chapter 8's".format(2 * n_two_sample(0.01) / n_paired(0.01, 0.12))
)
print(
" design replays identical deals rather than comparing two players'"
)
print(
" results, and it is the only way the trial fits inside a feasible"
" budget."
)
print(
"\nD. A warning about the paired design. Replaying a deal under two"
" policies"
)
print(
" changes what the opponents see, so after the first differing"
" decision"
)
print(
" the two replays are no longer the same game. The variance"
" reduction is"
)
print(
" therefore real but partial, and it decays with the length of the"
" hand."
)
print(
" Chapter 8 handles this by scoring only the FIRST differing"
" decision per"
)
print(
" hand against a fixed opponent policy, which keeps the pairing"
" exact at"
)
print(" the cost of discarding most of each hand.")
# Per-hand outcome s.d. 0.435 buy-ins.
#
# A. Hands needed per arm to detect an improvement of d per hand,
# two independent samples, two-sided 0.05, 90 % power:
# d per hand as a share of the s.d. hands per arm hands in total
# 0.100 0.230 398 796
# 0.050 0.115 1,591 3,182
# 0.020 0.046 9,942 19,884
# 0.010 0.023 39,766 79,532
# 0.005 0.011 159,062 318,124
# 0.002 0.005 994,134 1,988,268
#
# A 0.01 improvement per hand is a large edge in this game, and detecting
# it takes about 39,766 hands per arm. At 60 hands in a session that is
# about 663 sessions - which is why individual experience is almost
# worthless as evidence about strategy, and why the confident opinions
# that players hold about marginal decisions are not supportable by the
# number of hands they have played.
#
# B. What a player can actually learn from a season. With 10,000 hands:
# the smallest improvement detectable at 90 % power is
# 0.0199 buy-ins per hand (4.6% of the outcome s.d.).
# Anything smaller is invisible at that sample size, however real.
#
# C. The variance reduction that makes it feasible. If the same hands are
# replayed under both policies - the duplicate-game design of Chapter 8 -
# the deal is held fixed and only the decisions differ, so the comparison
# is paired and the relevant s.d. is that of the DIFFERENCE:
# s.d. of the difference as a share of the raw s.d. hands for d = 0.01
# 0.420 96.6% 18,536
# 0.300 69.0% 9,457
# 0.200 46.0% 4,203
# 0.120 27.6% 1,514
# 0.080 18.4% 673
#
# If duplicating the deals cuts the standard deviation of the difference
# to 0.12, the same effect needs 1,514 hands instead of 79,532, a
# reduction of about 53-fold. This is the whole reason Chapter 8's
# design replays identical deals rather than comparing two players'
# results, and it is the only way the trial fits inside a feasible budget.
#
# D. A warning about the paired design. Replaying a deal under two policies
# changes what the opponents see, so after the first differing decision
# the two replays are no longer the same game. The variance reduction is
# therefore real but partial, and it decays with the length of the hand.
# Chapter 8 handles this by scoring only the FIRST differing decision per
# hand against a fixed opponent policy, which keeps the pairing exact at
# the cost of discarding most of each hand.Section A is the book’s most useful and least comfortable table. An improvement of 0.01 buy-ins a hand is a large edge in this game, and detecting it with a two-sample test takes about 39,766 hands per arm, roughly 663 sessions. So
personal experience is almost incapable of serving as evidence about strategy.
Section B gives the calculation in reverse: 10,000 hands can only detect improvements above 0.0199 buy-ins a hand, and anything smaller is invisible in that sample however real it is.
Section C gives the step that makes a trial feasible: a duplicate design. The same deals are replayed under both strategies, and after pairing the relevant standard deviation is that of the difference. If replay brings it down to 0.12, the required number falls from 79,532 to 1,514, about 53-fold.
Section D states the design’s cost: the two replays are no longer the same game after the first point at which the strategies diverge, so the variance reduction is real but incomplete, and it decays with the length of the hand. This protocol’s response is to score only the decision at the first divergence, scoring against fixed opponent strategies so that the pairing stays exact — at the cost of discarding most of every hand.
Part three: the pre-registered strategy comparison
Design choices
First, one primary comparison only: strategy B against strategy A, in a duplicate design.
Second, one primary outcome only: the mean points difference per scored decision, in buy-ins.
Third, the opponent pool is fixed and published. Following Chapter 6, this book makes no claim about equilibrium or unexploitability: the conclusion holds only for the pool used, and a different pool is a different experiment.
Fourth, score only the decision at the first divergence, so the pairing stays exact (section D above).
Fifth, the protocol, the code and the decision rule are publicly registered before any data are generated (Nosek et al. 2018), and released following reproducible-computing practice (Wilson et al. 2014).
Sixth, conducted and analysed by researchers independent of the author’s businesses, with no wagering of real money.
Protocol
| Element | Specification |
|---|---|
| Conduct and registration | Carried out by a research team unconnected with the author; protocol, analysis code and decision rule publicly registered before the first deal is generated (Nosek et al. 2018); reported against the applicable CONSORT items (Schulz, Altman, and Moher 2010) |
| Subjects | Two software strategies; no human players, no wagering of real money |
| Deals | A pre-generated, fixed sequence of deals, replayed once under each strategy, with published random seeds |
| Opponents | A fixed, published opponent pool (several rule-based and several learning agents), unchanged for the whole run. A change to the pool constitutes a gate failure |
| Scoring | Only the decision at which the two strategies first diverge is scored; after that decision the hand is played out to its end by the fixed opponent strategies |
| Primary outcome | Mean points difference per scored decision (buy-ins), compared pairwise |
| Smallest effect of interest | 0.010 buy-ins per scored decision |
| Secondary outcomes | Win rate, deal-in rate, mean placement, and the components of the points total. All descriptive |
| Pre-specified qualifiers | The proportion of decisions on which the strategies agree (too high and the comparison concerns only a rare situation); the share of the total difference contributed by the largest 1 % of hands (too high and trimmed means and bootstrap intervals are reported alongside) |
| Equivalence reading | If the primary effect falls within ± the smallest effect of interest, it is reported under an equivalence framework as “no difference worth changing practice for” (Lakens, Scheel, and Isager 2018) |
Sample size
Calculation
# Chapter 8. Sample size for the pre-registered policy comparison. Design: a
# duplicate design in which the SAME deals are replayed with two decision
# policies, scored at the first decision at which the policies differ, against
# a fixed opponent pool. Primary
# outcome: the mean points difference per scored decision. The pairing is what
# makes the trial feasible (p11); the program sizes it on the paired standard
# deviation and then checks the normal-theory answer against a simulation with
# a realistic, heavy-tailed outcome distribution, because mahjong scores are
# not normal and the t test's behaviour in the tail matters.
import math
import random
from statistics import NormalDist
Z = NormalDist().inv_cdf
SESOI = 0.010 # buy-ins per scored decision
SD_DIFF = 0.120 # paired s.d. from the duplicate design (p11)
def n_paired(d=SESOI, sd=SD_DIFF, power=0.90, alpha=0.05):
return (Z(1 - alpha / 2) + Z(power)) ** 2 * sd ** 2 / d ** 2
# a heavy-tailed paired difference: mostly zero, occasionally a large swing
def draw_diff(rng, delta):
u = rng.random()
if u < 0.70:
return delta + rng.gauss(0.0, 0.02) # small or no consequence
if u < 0.94:
return delta + rng.gauss(0.0, 0.12) # a normal-sized swing
# a big hand changes hands
return delta + rng.gauss(0.0, 0.45)
def one_trial(rng, n, delta):
xs = [draw_diff(rng, delta) for _ in range(n)]
m = math.fsum(xs) / n
v = math.fsum((x - m) ** 2 for x in xs) / (n - 1)
se = math.sqrt(v / n)
t = m / se if se > 0 else 0.0
return t > Z(0.975), m
if __name__ == "__main__":
print(
"Duplicate design: the same deals replayed under two policies,"
" scored at"
)
print("the first differing decision, against a fixed opponent pool.")
print("SESOI {:.3f} buy-ins per scored decision; paired s.d. {:.3f}.\n"
.format(SESOI, SD_DIFF))
n0 = n_paired()
print("normal theory: n = {:,.1f} -> {:,} scored decisions"
.format(n0, math.ceil(n0)))
rng = random.Random(5514)
# empirical s.d. of the heavy-tailed generator, for comparison
samp = [draw_diff(rng, 0.0) for _ in range(200000)]
m = math.fsum(samp) / len(samp)
sd_emp = math.sqrt(
math.fsum((x - m) ** 2 for x in samp) / (len(samp) - 1)
)
print("generator's actual s.d.: {:.4f} (the planning value was {:.3f})"
.format(sd_emp, SD_DIFF))
n_adj = n_paired(sd=sd_emp)
print("normal theory at the generator's s.d.: {:,} scored decisions"
.format(math.ceil(n_adj)))
print("\nsimulation with the heavy-tailed paired difference:")
chosen = None
for n in (math.ceil(n_adj), 12000, 15000, 18000):
res = [one_trial(rng, n, SESOI) for _ in range(600)]
pw = sum(r[0] for r in res) / len(res)
se = math.sqrt(pw * (1 - pw) / len(res))
print(" n = {:6,}: power {:.3f} (s.e. {:.3f})".format(n, pw, se))
if chosen is None and pw - 2 * se >= 0.90:
chosen = n
if chosen is None:
chosen = 18000
t1 = sum(one_trial(rng, chosen, 0.0)[0] for _ in range(600)) / 600
print(" smallest n with power - 2 s.e. >= 0.90: {:,}".format(chosen))
print(
" one-sided error rate with no effect: {:.3f} (nominal 0.025)"
.format(t1)
)
print(
"\nplanned: {:,} scored decisions. At roughly one scored decision per"
" hand".format(chosen)
)
print(
"and 60 hands per session, that is about {:,.0f} sessions of play, or"
.format(chosen / 60)
)
print("a few weeks of continuous self-play for a software agent.")
print("\nsensitivity (normal theory):")
print(" SESOI \\ paired s.d. 0.08 0.12 0.20")
for d in (0.005, 0.010, 0.020):
print(" {:.3f} buy-ins ".format(d) + "".join(
"{:10,.0f}".format(math.ceil(n_paired(d=d, sd=s)))
for s in (0.08, 0.12, 0.20)))
print(
"\nTwo things this sizing makes explicit. First, a human cannot run"
" this"
)
print(
"trial: {:,} scored decisions is beyond what anyone plays"
" deliberately, so".format(chosen)
)
print(
"the comparison belongs to software agents replaying stored deals,"
" and the"
)
print(
"conclusions then apply to the policies rather than to people."
" Second, the"
)
print(
"SESOI of {:.3f} buy-ins per decision is small in absolute terms and"
" still".format(SESOI)
)
print(
"needs this many observations, which is the quantitative reason the"
" strategy"
)
print(
"debates in this game remain unsettled: the sample sizes required"
" have"
)
print("rarely been assembled.")
# Duplicate design: the same deals replayed under two policies, scored at
# the first differing decision, against a fixed opponent pool.
# SESOI 0.010 buy-ins per scored decision; paired s.d. 0.120.
#
# normal theory: n = 1,513.1 -> 1,514 scored decisions
# generator's actual s.d.: 0.1256 (the planning value was 0.120)
# normal theory at the generator's s.d.: 1,657 scored decisions
#
# simulation with the heavy-tailed paired difference:
# n = 1,657: power 0.880 (s.e. 0.013)
# n = 12,000: power 1.000 (s.e. 0.000)
# n = 15,000: power 1.000 (s.e. 0.000)
# n = 18,000: power 1.000 (s.e. 0.000)
# smallest n with power - 2 s.e. >= 0.90: 12,000
# one-sided error rate with no effect: 0.033 (nominal 0.025)
#
# planned: 12,000 scored decisions. At roughly one scored decision per hand
# and 60 hands per session, that is about 200 sessions of play, or
# a few weeks of continuous self-play for a software agent.
#
# sensitivity (normal theory):
# SESOI \ paired s.d. 0.08 0.12 0.20
# 0.005 buy-ins 2,690 6,053 16,812
# 0.010 buy-ins 673 1,514 4,203
# 0.020 buy-ins 169 379 1,051
#
# Two things this sizing makes explicit. First, a human cannot run this
# trial: 12,000 scored decisions is beyond what anyone plays deliberately, so
# the comparison belongs to software agents replaying stored deals, and the
# conclusions then apply to the policies rather than to people. Second, the
# SESOI of 0.010 buy-ins per decision is small in absolute terms and still
# needs this many observations, which is the quantitative reason the strategy
# debates in this game remain unsettled: the sample sizes required have
# rarely been assembled.Normal theory gives 1,514 scored decisions at a paired standard deviation of 0.120; this book checks that against a heavy-tailed generator of paired differences, because points in this game are not normal — most decisions have small consequences and an occasional large hand changes who wins. The simulation’s answer is larger than normal theory’s, and taking “power less two standard errors not below 0.90” gives 12,000 scored decisions.
At roughly one scored decision per hand and 60 hands a session, that is about 200 sessions, which for software agents is a few weeks of self-play. Two things follow and are stated plainly: a human cannot run this trial, so the comparison belongs to software agents and its conclusions apply to strategies rather than to people; and that a smallest effect of interest as small as 0.010 buy-ins a decision still needs this many observations is the quantified reason why strategic arguments about this game go unsettled for so long.
Decision rule
Calculation
# Chapter 8. Decision rule, fixed before the data are examined. Primary
# comparison: policy B against policy A in the duplicate design. D = mean
# points difference per scored decision, in buy-ins, with its 95 % confidence
# interval. POSITIVE D favours B. SESOI 0.010 buy-ins per scored decision.
# Gates, checked first: fewer than the planned number of scored decisions; the
# opponent pool changed during the run; the two policies' scored decisions not
# drawn from the same deals (pairing broken). Pre-specified qualifiers: the
# share of scored decisions in which the policies agreed (if very high, the
# comparison concerns a rare situation and does not generalise), and the
# heavy-tail check (if a small number of hands dominate the total, the
# interval is unreliable and a trimmed or bootstrap interval is reported too).
SESOI = 0.010
def verdict(d, lo, hi, n_scored, n_planned=12000, pool_changed=False,
pairing_broken=False, agree_share=0.5, top1_share=0.05):
for bad, why in (
(
pairing_broken,
(
"pairing broken: the two policies were not"
" scored on the same deals"
),
),
(pool_changed, "the opponent pool changed during the run"),
(
n_scored < 0.9 * n_planned,
"only {:,} of {:,} planned scored decisions".format(
n_scored, n_planned
),
),
):
if bad:
return "D = {:+.4f}: NOT INTERPRETABLE -\n {}".format(d, why)
if lo >= SESOI:
v = "B better by at least the SESOI"
elif lo > 0:
v = "B better, but not shown to reach the SESOI"
elif hi < 0:
v = "A better"
elif lo >= -SESOI and hi <= SESOI:
v = "no difference worth acting on"
else:
v = "unclear:\n the interval is too wide to decide"
notes = []
if agree_share > 0.95:
notes.append("the policies agreed on {:.0%} of decisions, so this"
"\n compares a rare situation and does not describe"
" overall play"
.format(agree_share))
if top1_share > 0.20:
notes.append(
"the largest 1 % of hands account for {:.0%} of the total"
"\n difference: report the trimmed and bootstrap"
" intervals too".format(top1_share)
)
out = "D = {:+.4f} ({:+.4f} to {:+.4f}): {}".format(d, lo, hi, v)
for nt in notes:
out += "\n qualifier: " + nt
return out
CASES = [
(0.0180, 0.0112, 0.0248, 12000),
(0.0062, 0.0014, 0.0110, 12000),
(0.0008, -0.0040, 0.0056, 12000),
(-0.0140, -0.0205, -0.0075, 12000),
(0.0090, -0.0180, 0.0360, 12000),
(0.0150, 0.0082, 0.0218, 12000, 12000, False, False, 0.985),
(0.0150, 0.0082, 0.0218, 12000, 12000, False, False, 0.5, 0.42),
(0.0200, 0.0130, 0.0270, 8000),
(0.0200, 0.0130, 0.0270, 12000, 12000, True),
(0.0200, 0.0130, 0.0270, 12000, 12000, False, True),
]
if __name__ == "__main__":
for c in CASES:
print(verdict(*c))
print("\nSecondary outcomes (win rate, deal-in rate, mean placement, the")
print(
"components of the score) are reported with intervals and never"
" change the"
)
print(
"verdict. No claim is made about equilibrium or unexploitability: per"
" p08,"
)
print(
"the result is a comparison against the FIXED opponent pool used,"
" and a"
)
print("different pool is a different experiment.")
print(
"\nThe example numbers above are invented, to show what each branch"
" prints."
)
# D = +0.0180 (+0.0112 to +0.0248): B better by at least the SESOI
# D = +0.0062 (+0.0014 to +0.0110): B better, but not shown to reach the SESOI
# D = +0.0008 (-0.0040 to +0.0056): no difference worth acting on
# D = -0.0140 (-0.0205 to -0.0075): A better
# D = +0.0090 (-0.0180 to +0.0360): unclear:
# the interval is too wide to decide
# D = +0.0150 (+0.0082 to +0.0218): B better, but not shown to reach the SESOI
# qualifier: the policies agreed on 98% of decisions, so this
# compares a rare situation and does not describe overall play
# D = +0.0150 (+0.0082 to +0.0218): B better, but not shown to reach the SESOI
# qualifier: the largest 1 % of hands account for 42% of the total
# difference: report the trimmed and bootstrap intervals too
# D = +0.0200: NOT INTERPRETABLE -
# only 8,000 of 12,000 planned scored decisions
# D = +0.0200: NOT INTERPRETABLE -
# the opponent pool changed during the run
# D = +0.0200: NOT INTERPRETABLE -
# pairing broken: the two policies were not scored on the same deals
#
# Secondary outcomes (win rate, deal-in rate, mean placement, the
# components of the score) are reported with intervals and never change the
# verdict. No claim is made about equilibrium or unexploitability: per p08,
# the result is a comparison against the FIXED opponent pool used, and a
# different pool is a different experiment.
#
# The example numbers above are invented, to show what each branch prints.There are three gates: fewer than nine tenths of the planned scored decisions; a change to the opponent pool during the run; and broken pairing (the two strategies not scored on the same deals). If any gate fails, the result is written “uninterpretable” whatever the primary outcome. There are two qualifiers: too high a proportion of agreement between the strategies (the comparison then concerns a rare situation and does not describe overall play); and a few hands dominating the total difference (trimmed means and bootstrap intervals are then reported alongside).
What this comparison cannot show
First, it compares two strategies only, on one fixed opponent pool, under one fixed set of rules. Second, it makes no claim about equilibrium or unexploitability (Chapter 6). Third, it scores only the first divergent decision, so it does not answer questions about “how to play a whole hand”. Fourth, it involves no human players, so it says nothing about whether any person can execute the strategy. Fifth, if the result is “no difference worth changing practice for”, that is the conclusion, and secondary outcomes cannot be substituted for it.
Part four: the section about money
The last section is unlike the others: it is not strategy but arithmetic.
Calculation
# Chapter 8. The arithmetic of playing for money, written out because a book
# of strategy is incomplete without it. Everything in Chapters 1 to 7 is about
# the game as a game. This programme answers a different question: for a
# player with a given edge, fee and stake, what does the distribution of
# outcomes look like over a year, and what fraction of otherwise identical
# players end the year down? The answer is used in Chapter 8 to state plainly
# what the book does and does not support, and it is the reason the book
# recommends studying the game rather than financing the study by playing it.
import math
import random
from statistics import NormalDist
N = NormalDist()
OUTCOMES = [(+0.90, 0.155), (+0.30, 0.110), (-0.10, 0.450), (-0.35, 0.225),
(-0.60, 0.060)]
def with_fee(fee):
return [(x - fee, p) for x, p in OUTCOMES]
def mean(d):
return math.fsum(x * p for x, p in d)
def sd(d):
m = mean(d)
return math.sqrt(math.fsum(p * (x - m) ** 2 for x, p in d))
def year_stats(hands, stake, fee):
d = with_fee(fee)
mu = mean(d) * stake * hands
s = sd(d) * stake * math.sqrt(hands)
return mu, s, N.cdf(-mu / s) if s > 0 else (0.0 if mu > 0 else 1.0)
if __name__ == "__main__":
HANDS = 60 * 52 # one session a week for a year
print(
"One session of 60 hands a week for a year: {:,} hands.".format(HANDS)
)
print("Stake 1 buy-in per hand throughout.\n")
print(
"A. A skilled player, before fees, has the edge of the p09"
" distribution:"
)
print(
" fee per hand expected year s.d. of the year P(down over the"
" year)"
)
for fee in (0.000, 0.010, 0.020, 0.040, 0.073):
mu, s, p = year_stats(HANDS, 1.0, fee)
print(" {:12.3f} {:15.1f} {:18.1f} {:23.3f}".format(fee, mu, s, p))
print(
"\n Note the last column at a fee that still leaves a positive"
" edge. At a"
)
mu, s, p = year_stats(HANDS, 1.0, 0.010)
print(
" fee of 0.010 the expected year is {:+.1f} buy-ins and yet"
" {:.0%} of".format(mu, p)
)
print(
" players with exactly this edge finish the year down. A positive"
)
print(
" expectation does not make a year of play likely to be"
" profitable; it"
)
print(
" makes it profitable on average over far more hands than a year"
" holds."
)
print(
"\nB. How many hands before the probability of being down falls below"
" 5 %?"
)
for fee in (0.000, 0.010, 0.020, 0.040):
d = with_fee(fee)
mu1, s1 = mean(d), sd(d)
if mu1 <= 0:
print(" fee {:.3f}: never (no edge)".format(fee))
continue
# need mu1 n / (s1 sqrt(n)) >= 1.645
n_need = (1.645 * s1 / mu1) ** 2
print(" fee {:.3f}: {:,.0f} hands ({:,.0f} years at this rate)"
.format(fee, math.ceil(n_need), n_need / HANDS))
print(
"\n At a fee of 0.02 a player needs on the order of a decade of"
" weekly"
)
print(
" play before a profit is more likely than not by this standard,"
" and that"
)
print(
" is for a player whose edge is as large as the p09 distribution"
" assumes"
)
print(
" and who does not get worse, tired or unlucky in their choice of"
" table."
)
print(
"\nC. And for a player with NO edge, which is the position of most"
" people:"
)
for fee in (0.000, 0.010, 0.020):
d = [(x - fee, p) for x, p in OUTCOMES]
d = [(x - mean(d), p) for x, p in d] # remove the edge exactly
d = [(x - fee, p) for x, p in d]
mu, s = mean(d) * HANDS, sd(d) * math.sqrt(HANDS)
print(" fee {:.3f}: expected year {:+.1f} buy-ins, s.d. {:.1f},"
" P(down) {:.3f}".format(fee, mu, s, N.cdf(-mu / s)))
print(
"\n With no edge and any fee at all, the expected year is negative"
" and the"
)
print(
" probability of being down exceeds a half, by construction. There"
" is no"
)
print(
" staking rule, bankroll or discipline that changes this:"
" Chapter 6's"
)
print(
" Kelly result is explicitly zero stake when the edge is not"
" positive."
)
print(
"\nD. What the book therefore says. The models in Chapters 1 to 7 are"
" about"
)
print(
" a game of imperfect information, and they are worth studying as"
" that."
)
print(
" They do not establish that any reader has an edge, and"
" programme p11"
)
print(
" showed that establishing one takes more hands than most players"
" will"
)
print(
" ever record. Playing for money on the strength of an unmeasured"
" edge is"
)
print(
" the error this programme quantifies, and the risk is not"
" symmetric: the"
)
print(" losses are real and the edge is hypothetical.")
print(
"\n Gambling can cause serious financial and personal harm, and"
" it can"
)
print(
" become compulsive. Anyone who finds that they are playing more"
" than"
)
print(
" they intended, staking more than they can afford, or chasing"
" losses"
)
print(
" should treat that as the signal it is and seek help from a"
" qualified"
)
print(" service in their own country. This book is a study of a game's")
print(
" mathematics; it is not an inducement to bet on it, and the author"
)
print(" recommends studying the game without staking money on it.")
# One session of 60 hands a week for a year: 3,120 hands.
# Stake 1 buy-in per hand throughout.
#
# A. A skilled player, before fees, has the edge of the p09 distribution:
# fee per hand expected year s.d. of the year P(down over the year)
# 0.000 39.8 24.3 0.051
# 0.010 8.6 24.3 0.362
# 0.020 -22.6 24.3 0.824
# 0.040 -85.0 24.3 1.000
# 0.073 -188.0 24.3 1.000
#
# Note the last column at a fee that still leaves a positive edge. At a
# fee of 0.010 the expected year is +8.6 buy-ins and yet 36% of
# players with exactly this edge finish the year down. A positive
# expectation does not make a year of play likely to be profitable; it
# makes it profitable on average over far more hands than a year holds.
#
# B. How many hands before the probability of being down falls below 5 %?
# fee 0.000: 3,146 hands (1 years at this rate)
# fee 0.010: 67,611 hands (22 years at this rate)
# fee 0.020: never (no edge)
# fee 0.040: never (no edge)
#
# At a fee of 0.02 a player needs on the order of a decade of weekly
# play before a profit is more likely than not by this standard, and that
# is for a player whose edge is as large as the p09 distribution assumes
# and who does not get worse, tired or unlucky in their choice of table.
#
# C. And for a player with NO edge, which is the position of most people:
# fee 0.000: expected year -0.0 buy-ins, s.d. 24.3, P(down) 0.500
# fee 0.010: expected year -31.2 buy-ins, s.d. 24.3, P(down) 0.901
# fee 0.020: expected year -62.4 buy-ins, s.d. 24.3, P(down) 0.995
#
# With no edge and any fee at all, the expected year is negative and the
# probability of being down exceeds a half, by construction. There is no
# staking rule, bankroll or discipline that changes this: Chapter 6's
# Kelly result is explicitly zero stake when the edge is not positive.
#
# D. What the book therefore says. The models in Chapters 1 to 7 are about
# a game of imperfect information, and they are worth studying as that.
# They do not establish that any reader has an edge, and programme p11
# showed that establishing one takes more hands than most players will
# ever record. Playing for money on the strength of an unmeasured edge is
# the error this programme quantifies, and the risk is not symmetric: the
# losses are real and the edge is hypothetical.
#
# Gambling can cause serious financial and personal harm, and it can
# become compulsive. Anyone who finds that they are playing more than
# they intended, staking more than they can afford, or chasing losses
# should treat that as the signal it is and seek help from a qualified
# service in their own country. This book is a study of a game's
# mathematics; it is not an inducement to bet on it, and the author
# recommends studying the game without staking money on it.The point of section A is in its last column. At a fee of 0.010, with an expected annual return of buy-ins, 36 % of players still end the year down. A positive expectation does not make a year of play likely to profit; it makes play profitable on average over far more hands than a year contains.
Section B gives “how long until losing is unlikely”: at a fee of 0.010 the number of hands required is more than a year’s worth, and that is for a player whose edge is as large as this book’s distribution, and who does not get worse, does not tire and does not pick the wrong table.
Section C gives most people’s situation: without an edge, any fee makes the expectation negative and the probability of losing greater than a half. Chapter 6’s Kelly result is unambiguous here — the optimal stake when the edge is not positive is zero.
Section D is the book’s position. The models of Chapters 1 to 7 are about an imperfect-information game, and as that they are worth studying. They do not establish that any reader has an edge, and program p11 shows that establishing one takes more hands than most players will ever record. Staking money on an unmeasured edge is the error this section quantifies, and the risk is asymmetric: the losses are real and the edge is hypothetical.
Note
Gambling-related harm. Gambling can cause serious financial and personal harm and can become compulsive; there are systematic reviews of the prevalence of and factors associated with problem gambling (Calado and Griffiths 2016), and dedicated reviews of gambling disorder as a diagnostic entity (Potenza et al. 2019). Anyone who finds themselves playing more than intended, staking more than they can afford, or chasing losses should treat that as a signal and seek help from a qualified service in their own country. This book’s recommendation is to study the game and not to stake money on it.
Disclosure
The author uses the 5D Library volumes, including this one, as the methodological basis of his community courses, 5D Ascension Cards (Dimensional Shift Card Decks) and the Yafu AI knowledge base, and therefore has a commercial interest in this book’s subject matter. This book promotes no course, card deck, software or service; it provides no link to or recommendation of any gambling platform, agent or financial service, and accepts no sponsorship from any such organisation. It is not a gambling guide: it does not advise anyone to stake money on the game it discusses, the last section of Chapter 8 uses a calculation to show why staking on an unmeasured edge is a quantifiable error, and it notes that gambling can cause serious financial and personal harm and can become compulsive. The trial in Chapter 8 is a comparison of strategies between software agents, conducted and analysed by researchers independent of the author’s businesses, involving no wagering of real money and recruiting no one from the author’s community, students or clients; all code and research protocols in the book are public and open to anyone’s inspection.
Summary
The four-step checklist reads a strategic claim along four lines: which class of quantity, direction, who the opponent is, and sample. Applied to the usual reasoning it shows that the two classes are mixed, that at least three links point the wrong way, that the opponent pool is usually unstated, and that confident opinions about marginal decisions are not supported by the number of hands. Sample size is the least comfortable part of the chapter: an improvement of 0.01 buy-ins a hand needs about 39,766 hands per arm, and 10,000 hands can only detect improvements above 0.0199; a duplicate design cuts the requirement about 53-fold, at the cost of scoring only the decision at the first divergence. The pre-registered comparison has strategy B against strategy A as its single primary comparison and the mean points difference per scored decision as its single primary outcome, with a smallest effect of interest of 0.010 buy-ins, requiring 12,000 scored decisions, a decision rule written as code, and the agreement proportion and the share contributed by a few hands as pre-specified qualifiers; it makes no claim about equilibrium. The last section computes that at a fee of 0.010 an expected annual return of buy-ins still leaves 36 % of players down at the end of the year, that without an edge any fee makes the expectation negative, and that the optimal stake when the edge is not positive is zero.
References
Fan, Y. (2026). Imperfect-Information Stochastic Games, Bayesian Hand-Reading and Counterfactual Regret Minimisation in Four-Player Tile Dynamics: From Markov Transitions of the Tile Pool and the Optimal Stopping of Defensive Folding to Kelly Allocation in a Multiplayer Game, with a Textual Examination of the Orthodox and the Unorthodox in the Classical Military Writers [四人不完全信息随机博弈、贝叶斯读牌与反事实遗憾最小化] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24020464 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).