5D Library › Imperfect-Information Stochastic Games, Bayesian Hand-Reading and Counterfactual Regret Minimisation in Four-Player Tile Dynamics · 中文
Chapter 5. Folding: Optimal Stopping, and a Conclusion Stated Backwards
Written as optimal stopping
One opponent is visibly ready. From then on, each turn the player either pushes (discards a tile that may deal in, keeping their own chance alive) or folds (discards only safe tiles thereafter, abandoning their hand but no longer able to deal in).
This is a finite-horizon Markov decision process (Puterman 1994), solvable exactly by backward induction (Bellman 1957); the general theory of optimal stopping is in Chow, Robbins and Siegmund (Chow, Robbins, and Siegmund 1971), and Volume 42 covered this class of problem.
The key modelling choice: folding is absorbing. Once a player folds they stay folded for the rest of the hand, so the value of folding is just the opponent’s self-draw risk over the remaining turns.
Calculation
# Chapter 5. Folding as optimal stopping, and the limit of that framing. With
# one opponent visibly ready, each remaining turn a player either PUSHES
# (discards a tile that may deal in, keeping their own chance alive) or FOLDS
# (discards only safe tiles for the rest of the hand, abandoning their own
# hand but no longer able to deal in). Folding is absorbing: once a player
# folds they stay folded, so the value of folding at turn t is just the
# opponent's self-draw risk over the remaining turns. Pushing is solved by
# backward induction against that. Illustrative points; standard library only.
import math
def fold_value(t, turns, p_tsumo, v_tsumo):
"""Fold now and stay folded: lose v_tsumo if the opponent self-draws at
any
remaining turn."""
remaining = turns - t
return -(1.0 - (1.0 - p_tsumo) ** remaining) * v_tsumo
def solve(turns, p_win, v_win, p_deal, v_deal, p_tsumo, v_tsumo):
V = [0.0] * (turns + 1)
policy = [None] * turns
for t in range(turns - 1, -1, -1):
# push this turn: own win, deal-in, opponent self-draw, or continue
p_cont = 1.0 - p_win - p_deal - p_tsumo
push = (p_win * v_win - p_deal * v_deal - p_tsumo * v_tsumo
+ p_cont * V[t + 1])
fold = fold_value(t, turns, p_tsumo, v_tsumo)
if push >= fold:
V[t], policy[t] = push, "push"
else:
V[t], policy[t] = fold, "fold"
return V, policy
def first_push_turn(policy):
"""Turns are indexed 0..turns-1 from now; report the first turn at which
pushing is optimal, or None."""
for i, a in enumerate(policy):
if a == "push":
return i + 1
return None
if __name__ == "__main__":
TURNS = 12
VW, VD, VT = 3900, 5200, 2600
PT = 0.03
print("One opponent visibly ready; {} turns remain.".format(TURNS))
print("Own win +3900, dealing in -5200,")
print(
"opponent self-draw -2600 at {:.0%} per turn. Folding is absorbing.\n"
.format(PT)
)
print(
"A. Own hand ready, two-sided wait (p_win 0.10 per turn), deal-in"
" risk 0.05:"
)
V, pol = solve(TURNS, 0.10, VW, 0.05, VD, PT, VT)
print(" turn from now policy value from here fold value here")
for t in range(TURNS):
print(" {:13d} {:6s} {:16.1f} {:17.1f}"
.format(t + 1, pol[t], V[t], fold_value(t, TURNS, PT, VT)))
print(
"\nB. Policy against own win probability, deal-in risk fixed at 0.05:"
)
print(" p_win turns pushed (of {}) value now".format(TURNS))
for pw in (0.02, 0.04, 0.06, 0.08, 0.12, 0.16):
V, pol = solve(TURNS, pw, VW, 0.05, VD, PT, VT)
print(" {:5.2f} {:22d} {:12.1f}"
.format(pw, sum(1 for a in pol if a == "push"), V[0]))
print(
"\nC. Policy against deal-in risk, own win probability fixed at 0.08:"
)
print(" p_deal turns pushed (of {}) value now".format(TURNS))
for pd in (0.01, 0.03, 0.05, 0.08, 0.12):
V, pol = solve(TURNS, 0.08, VW, pd, VD, PT, VT)
print(" {:6.2f} {:22d} {:12.1f}"
.format(pd, sum(1 for a in pol if a == "push"), V[0]))
print(
"\nD. The one-turn indifference condition. Ignoring continuation,"
" pushing"
)
print("beats folding when p_win x {} >= p_deal x {},".format(VW, VD))
print("i.e. p_win >= p_deal x {:.3f}.".format(VD / VW))
print(
" (the opponent's self-draw term cancels: folding does not"
" prevent it)"
)
print(" p_deal break-even p_win")
for pd in (0.01, 0.03, 0.05, 0.08, 0.12):
print(" {:6.2f} {:18.3f}".format(pd, pd * VD / VW))
print(
"\n The self-draw term cancelling is the structural point, and it"
" is often"
)
print(
" stated backwards. Folding buys protection ONLY against dealing"
" in; it"
)
print(
" buys nothing against the opponent drawing the tile themselves."
" So the"
)
print(
" case for folding rests entirely on p_deal, and a nearly safe tile"
" should"
)
print(" be pushed even with a weak hand.")
print(
"\nE. What this framing gets WRONG. Three things, in order of"
" seriousness."
)
print(
" 1. It is not a decision problem at all. p_deal depends on"
" what the"
)
print(
" opponent waits on, and a thinking opponent chooses that"
" partly in"
)
print(
" anticipation of being read (p03). Treating p_deal as a"
" constant"
)
print(
" assumes a non-adaptive opponent, and Chapter 6 shows what"
" changes"
)
print(" when that assumption is dropped.")
print(
" 2. There are three opponents, not one, and their hands are"
" dependent"
)
print(
" through the tiles nobody has drawn. A tile safe against one"
" can be"
)
print(
" the winning tile for another, so 'fold' is not a single"
" action."
)
print(
" 3. Points are not utility. In the last hand of a match, with a"
" deficit"
)
print(
" to make up, the correct policy can be to take a"
" negative-expectation"
)
print(
" push, which this objective cannot express. Chapter 6"
" treats the"
)
print(
" bankroll version of that problem and Chapter 8 the match"
" version."
)
# One opponent visibly ready; 12 turns remain.
# Own win +3900, dealing in -5200,
# opponent self-draw -2600 at 3% per turn. Folding is absorbing.
#
# A. Own hand ready, two-sided wait (p_win 0.10 per turn), deal-in risk 0.05:
# turn from now policy value from here fold value here
# 1 push 262.2 -796.0
# 2 push 256.3 -740.2
# 3 push 249.2 -682.7
# 4 push 240.5 -623.4
# 5 push 229.8 -562.3
# 6 push 216.9 -499.2
# 7 push 201.1 -434.3
# 8 push 181.8 -367.3
# 9 push 158.3 -298.2
# 10 push 129.6 -227.1
# 11 push 94.6 -153.7
# 12 push 52.0 -78.0
#
# B. Policy against own win probability, deal-in risk fixed at 0.05:
# p_win turns pushed (of 12) value now
# 0.02 0 -796.0
# 0.04 0 -796.0
# 0.06 8 -609.8
# 0.08 12 -142.4
# 0.12 12 605.3
# 0.16 12 1147.4
#
# C. Policy against deal-in risk, own win probability fixed at 0.08:
# p_deal turns pushed (of 12) value now
# 0.01 12 1189.6
# 0.03 12 466.0
# 0.05 12 -142.4
# 0.08 2 -777.3
# 0.12 0 -796.0
#
# D. The one-turn indifference condition. Ignoring continuation, pushing
# beats folding when p_win x 3900 >= p_deal x 5200,
# i.e. p_win >= p_deal x 1.333.
# (the opponent's self-draw term cancels: folding does not prevent it)
# p_deal break-even p_win
# 0.01 0.013
# 0.03 0.040
# 0.05 0.067
# 0.08 0.107
# 0.12 0.160
#
# The self-draw term cancelling is the structural point, and it is often
# stated backwards. Folding buys protection ONLY against dealing in; it
# buys nothing against the opponent drawing the tile themselves. So the
# case for folding rests entirely on p_deal, and a nearly safe tile should
# be pushed even with a weak hand.
#
# E. What this framing gets WRONG. Three things, in order of seriousness.
# 1. It is not a decision problem at all. p_deal depends on what the
# opponent waits on, and a thinking opponent chooses that partly in
# anticipation of being read (p03). Treating p_deal as a constant
# assumes a non-adaptive opponent, and Chapter 6 shows what changes
# when that assumption is dropped.
# 2. There are three opponents, not one, and their hands are dependent
# through the tiles nobody has drawn. A tile safe against one can be
# the winning tile for another, so 'fold' is not a single action.
# 3. Points are not utility. In the last hand of a match, with a deficit
# to make up, the correct policy can be to take a negative-expectation
# push, which this objective cannot express. Chapter 6 treats the
# bankroll version of that problem and Chapter 8 the match version.The term that cancels
Section D is the chapter’s main result, and it is often stated backwards.
The one-turn indifference condition is that pushing beats folding exactly when
that is, . The opponent’s self-draw term does not appear in the condition — because folding does not prevent a self-draw, the term is equal on both sides and cancels. Hence
folding protects only against dealing in. The case for folding therefore rests entirely on the deal-in probability, and a nearly safe tile should be pushed even with a weak hand.
The common formulation is “when in doubt, fold”, whereas by the condition above, what to look at when in doubt is how dangerous this tile is, not how weak the hand is. The two give opposite advice when the hand is weak and the tile is safe, which is a common situation.
Sections B and C give the policy against the two parameters, and the direction is monotone: the higher one’s own win probability the longer one pushes, and the higher the deal-in probability the shorter. Section A gives a concrete trajectory with the fold value listed beside each turn, so that the comparison can be seen being made.
Three places where this framing fails
Section E states three, in order of seriousness.
First, it is not a decision problem at all. The deal-in probability depends on what the opponent waits on, and a thinking opponent chooses their wait and their discards partly in anticipation of being read (Chapter 3). Treating as a constant assumes a non-adaptive opponent, and Chapter 6 shows that against an adaptive one no fixed rule is safe.
Second, there are three opponents, not one. Their hands are dependent through the tiles nobody drew, so a tile safe against one may be exactly another’s winning tile. “Folding” is therefore not a single action — folding against one can feed another. This book’s model handles one opponent, and that is an explicit simplification.
Third, points are not utility. In the last hand of a match, with a deficit to make up, the correct policy can be to accept a negative-expectation push, because the objective is placement rather than points. This objective function cannot express that. Chapter 6 handles the same problem at the level of the bankroll (Kelly and ruin), and Chapter 8 treats it as the terminal condition it is.
Hypothesis · not established
The author’s hypothesis (not established): against a fixed opponent pool, a policy that decides push or fold by this chapter’s indifference condition — looking only at the deal-in probability and one’s own win probability, and not at how “weak” the hand feels — beats a policy that decides primarily on hand strength. What would show it to be wrong: in the duplicate design of Chapter 8, a 95 % confidence interval for that comparison that includes zero or runs the other way.
Note
Relation to Volume 42. Volume 42 covered Bayesian search and optimal stopping, where the object is a single-agent decision problem and the stopping rule has clean optimality results. The difference here is precisely the first limitation above: the “environment” responds, so the standard results of optimal stopping apply only after the opponent has been frozen. Transferring single-agent stopping theory directly to a game is the commonest overreach in this kind of analysis.
Summary
Writing folding as a finite-horizon Markov decision process and solving it by backward induction gives a threshold policy: the higher one’s own win probability the longer one pushes, and the higher the deal-in probability the shorter. The main result is a structural conclusion that is often stated backwards: the opponent’s self-draw term cancels between pushing and folding, because folding protects only against dealing in, so the indifference condition contains only the deal-in probability and one’s own win probability (here ). A nearly safe tile should therefore be pushed even with a weak hand, which is the opposite of “when in doubt, fold”. The framing fails in three places: the deal-in probability depends on an opponent who adapts, so this is not a decision problem; there are three opponents with dependent hands, so “fold” is not a single action; and points are not utility, so in the last hand of a match the correct policy can be a negative-expectation push.
References
Fan, Y. (2026). Imperfect-Information Stochastic Games, Bayesian Hand-Reading and Counterfactual Regret Minimisation in Four-Player Tile Dynamics: From Markov Transitions of the Tile Pool and the Optimal Stopping of Defensive Folding to Kelly Allocation in a Multiplayer Game, with a Textual Examination of the Orthodox and the Unorthodox in the Classical Military Writers [四人不完全信息随机博弈、贝叶斯读牌与反事实遗憾最小化] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24020464 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).