5D Library › 四人不完全信息随机博弈、贝叶斯读牌与反事实遗憾最小化 · English

第八章 实践与预注册检验

注

本章是研究方案与文献阅读方法,不是赌博指南。 第八章的试验是软件代理之间的策略比较,不涉及真实金钱投注。核查表帮助读者在读到一项策略说法时把问题问清楚。本章最后一节用计算说明为什么以未经测量的优势投注金钱是一个可量化的错误。

第一部分 四步核查表

前七章可以浓缩为一份四步核查表,用于阅读任何「某种打法更好」的说法。

  1. 哪一类量。 这个说法依赖的是只看自己手牌的量(向聴、进张、手牌价值),还是依赖对手手牌的量(放铳概率、对手的听牌)?前者可以精确计算,后者只能在模型与先验下估计(第一章)。
  2. 方向。 有没有哪一环的方向与它所依据的数学相反?数进张在前期高估宽听(第二章);忽略另外三家把胜率高估 1.36 到 1.79 倍且偏向慢而贵的手牌(第四章,p05);弃牌只防放铳所以自摸项相消(第五章)。
  3. 对手是谁。 这个说法假设对手适应还是不适应?读牌是对不适应对手的最优反应(第三章);混合的价值以对手的观察量为条件(第六章)。有没有把「对某个对手池更好」说成「更好」?
  4. 样本。 支持这个说法的手数是多少?按程序 p11,每手 0.01 个买入的改善需要每组约 39,766 手才能检出(第六章与本章)。个人经验是不是被当成了证据?

把核查表用于关于这个博弈的通行策略推理,结果如下表。

步骤 通行推理的情况 能支持的结论
哪一类量 两类混用,且常把第二类的估计当成第一类的计算 关于自己手牌的结论较可靠;关于对手的结论须连带先验一起引用
方向 至少三处方向相反(前期数进张、忽略竞争风险、「拿不准就弃」) 这几条需要改写而不是补充
对手是谁 多半不写明对手池 结论只对所假设的对手类型成立
样本 以个人经验与印象为主 边际决策上的自信意见不被手数支持

第二部分 需要多少手

在设计任何比较之前,先算它要多大。

推导

# Chapter 8. How many hands does it take to know whether a change helped? This
# is the single most useful number in the book, and it is uncomfortable. With
# a per-hand outcome standard deviation of s and a true improvement of d per
# hand, distinguishing d from zero at the usual error rates needs
#   n = 2 (z_{1-a/2} + z_{1-b})^2 s^2 / d^2      (two independent samples)
#   n = (z_{1-a/2} + z_{1-b})^2 s^2 / d^2        (one sample against a known
#   mean)
# and because s is large relative to any plausible d, n is in the thousands to
# hundreds of thousands. The program also computes the paired/variance-reduced
# version, which is the only thing that makes the measurement feasible at all.
import math
from statistics import NormalDist

Z = NormalDist().inv_cdf
# per-hand outcome s.d. in buy-ins, from the p09 distribution
SD_HAND = 0.435

def n_two_sample(d, sd=SD_HAND, power=0.90, alpha=0.05):
    return 2 * (Z(1 - alpha / 2) + Z(power)) ** 2 * sd ** 2 / d ** 2

def n_paired(d, sd_diff, power=0.90, alpha=0.05):
    return (Z(1 - alpha / 2) + Z(power)) ** 2 * sd_diff ** 2 / d ** 2

if __name__ == "__main__":
    print("Per-hand outcome s.d. {:.3f} buy-ins.\n".format(SD_HAND))

    print("A. Hands needed per arm to detect an improvement of d per hand,")
    print("two independent samples, two-sided 0.05, 90 % power:")
    print(
        "   d per hand   as a share of the s.d.   hands per arm   hands in"
        " total"
    )
    for d in (0.100, 0.050, 0.020, 0.010, 0.005, 0.002):
        n = n_two_sample(d)
        print("   {:10.3f} {:23.3f} {:15,.0f} {:16,.0f}"
              .format(d, d / SD_HAND, math.ceil(n), 2 * math.ceil(n)))

    print(
        "\n   A 0.01 improvement per hand is a large edge in this game, and"
        " detecting"
    )
    print(
        "   it takes about {:,.0f} hands per arm. At 60 hands in a session"
        " that is".format(math.ceil(n_two_sample(0.01)))
    )
    print(
        "   about {:,.0f} sessions - which is why individual experience is"
        " almost".format(math.ceil(n_two_sample(0.01)) / 60)
    )
    print(
        "   worthless as evidence about strategy, and why the confident"
        " opinions"
    )
    print(
        "   that players hold about marginal decisions are not supportable"
        " by the"
    )
    print("   number of hands they have played.")

    print(
        "\nB. What a player can actually learn from a season. With {:,}"
        " hands:".format(10000)
    )
    n = 10000
    print("   the smallest improvement detectable at 90 % power is")
    d_min = math.sqrt(2 * (Z(0.975) + Z(0.90)) ** 2 * SD_HAND ** 2 / n)
    print("   {:.4f} buy-ins per hand ({:.1%} of the outcome s.d.)."
          .format(d_min, d_min / SD_HAND))
    print(
        "   Anything smaller is invisible at that sample size, however real."
    )

    print(
        "\nC. The variance reduction that makes it feasible. If the same"
        " hands are"
    )
    print(
        "replayed under both policies - the duplicate-game design of"
        " Chapter 8 -"
    )
    print(
        "the deal is held fixed and only the decisions differ, so the"
        " comparison"
    )
    print("is paired and the relevant s.d. is that of the DIFFERENCE:")
    print(
        "   s.d. of the difference   as a share of the raw s.d.   hands for d"
        " = 0.01"
    )
    for sd_diff in (0.42, 0.30, 0.20, 0.12, 0.08):
        print("   {:22.3f} {:28.1%} {:19,.0f}"
              .format(sd_diff, sd_diff / SD_HAND,
                      math.ceil(n_paired(0.01, sd_diff))))
    print(
        "\n   If duplicating the deals cuts the standard deviation of the"
        " difference"
    )
    print(
        "   to {:.2f}, the same effect needs {:,.0f} hands instead of"
        " {:,.0f}, a".format(
            0.12,
            math.ceil(n_paired(0.01, 0.12)),
            2 * math.ceil(n_two_sample(0.01)),
        )
    )
    print(
        "   reduction of about {:.0f}-fold. This is the whole reason"
        " Chapter 8's".format(2 * n_two_sample(0.01) / n_paired(0.01, 0.12))
    )
    print(
        "   design replays identical deals rather than comparing two players'"
    )
    print(
        "   results, and it is the only way the trial fits inside a feasible"
        " budget."
    )

    print(
        "\nD. A warning about the paired design. Replaying a deal under two"
        " policies"
    )
    print(
        "   changes what the opponents see, so after the first differing"
        " decision"
    )
    print(
        "   the two replays are no longer the same game. The variance"
        " reduction is"
    )
    print(
        "   therefore real but partial, and it decays with the length of the"
        " hand."
    )
    print(
        "   Chapter 8 handles this by scoring only the FIRST differing"
        " decision per"
    )
    print(
        "   hand against a fixed opponent policy, which keeps the pairing"
        " exact at"
    )
    print("   the cost of discarding most of each hand.")
# Per-hand outcome s.d. 0.435 buy-ins.
#
# A. Hands needed per arm to detect an improvement of d per hand,
# two independent samples, two-sided 0.05, 90 % power:
#    d per hand   as a share of the s.d.   hands per arm   hands in total
#         0.100                   0.230             398              796
#         0.050                   0.115           1,591            3,182
#         0.020                   0.046           9,942           19,884
#         0.010                   0.023          39,766           79,532
#         0.005                   0.011         159,062          318,124
#         0.002                   0.005         994,134        1,988,268
#
#    A 0.01 improvement per hand is a large edge in this game, and detecting
#    it takes about 39,766 hands per arm. At 60 hands in a session that is
#    about 663 sessions - which is why individual experience is almost
#    worthless as evidence about strategy, and why the confident opinions
#    that players hold about marginal decisions are not supportable by the
#    number of hands they have played.
#
# B. What a player can actually learn from a season. With 10,000 hands:
#    the smallest improvement detectable at 90 % power is
#    0.0199 buy-ins per hand (4.6% of the outcome s.d.).
#    Anything smaller is invisible at that sample size, however real.
#
# C. The variance reduction that makes it feasible. If the same hands are
# replayed under both policies - the duplicate-game design of Chapter 8 -
# the deal is held fixed and only the decisions differ, so the comparison
# is paired and the relevant s.d. is that of the DIFFERENCE:
#    s.d. of the difference   as a share of the raw s.d.   hands for d = 0.01
#                     0.420                        96.6%              18,536
#                     0.300                        69.0%               9,457
#                     0.200                        46.0%               4,203
#                     0.120                        27.6%               1,514
#                     0.080                        18.4%                 673
#
#    If duplicating the deals cuts the standard deviation of the difference
#    to 0.12, the same effect needs 1,514 hands instead of 79,532, a
#    reduction of about 53-fold. This is the whole reason Chapter 8's
#    design replays identical deals rather than comparing two players'
#    results, and it is the only way the trial fits inside a feasible budget.
#
# D. A warning about the paired design. Replaying a deal under two policies
#    changes what the opponents see, so after the first differing decision
#    the two replays are no longer the same game. The variance reduction is
#    therefore real but partial, and it decays with the length of the hand.
#    Chapter 8 handles this by scoring only the FIRST differing decision per
#    hand against a fixed opponent policy, which keeps the pairing exact at
#    the cost of discarding most of each hand.

A 段是本书最有用也最不舒服的一张表。每手 0.01 个买入的改善在这个博弈里是很大的优势,而以两样本检验检出它需要每组约 39,766 手,即约 663 场。所以

个人经验几乎不能作为策略的证据。

B 段给出反向计算:10,000 手只能检出 0.0199 个买入/手以上的改善,比这更小的改善在那个样本上不可见,无论它多真实。

C 段给出使试验可行的那一步:重复对局设计。同样的牌局在两种策略下重放,配对后相关的标准差是差值的标准差。若重放把它降到 0.12,所需手数从 79,532 降到 1,514,约 53 倍。

D 段写明这个设计的代价:两次重放在两种策略首次分歧之后就不再是同一局博弈,所以方差缩减是真实但不完全的,而且随手牌长度衰减。本方案的处理是:只在首次分歧的决策上计分,对固定对手策略评分,从而保持配对严格——代价是丢掉每手牌的大部分。

第三部分 预注册的策略比较

设计选择

第一,只设一个主要比较:策略 B 对策略 A,在重复对局设计中。

第二,只设一个主要结局:每个计分决策的平均点数差,单位为买入。

第三,对手池固定并公开。按第六章,本书不作任何关于均衡或不可利用性的主张:结论只对所用的对手池成立,换一个池就是另一个实验。

第四,只在首次分歧的决策上计分,以保持配对严格(见上节 D 段)。

第五,方案、代码与判定规则在跑数据之前公开登记 (Nosek et al. 2018),并按可重复计算的惯例发布 (Wilson et al. 2014)。

第六,由独立于作者企业的研究人员实施与分析,且不涉及真实金钱投注。

方案

要素 规定
实施与登记 由与作者无关的研究团队实施;方案、分析代码与判定规则在第一手牌生成前公开登记 (Nosek et al. 2018);按 CONSORT 的适用条目报告 (Schulz, Altman, and Moher 2010)
对象 两个软件策略,不涉及人类玩家,不涉及真实金钱投注
牌局 预先生成并固定的牌局序列,两种策略各重放一次,随机种子公开
对手 固定且公开的对手池(规则型与学习型各若干),在整个运行期间不变。对手池改变即构成前置条件失败
计分 只在两种策略首次分歧的决策上计分;该决策之后由固定对手策略接续到这手牌结束
主要结局 每个计分决策的平均点数差(买入),配对比较
最小关注效应 0.010 个买入/计分决策
次要结局 和牌率、放铳率、平均顺位、点数的各组成部分。均为描述性
预先指定的限定条件 两策略一致的决策比例(过高则该比较只涉及一个罕见情形);最大 1% 的手牌占总差值的比例(过高则另报修尾与自助法区间)
等效性判读 若主要效应落在 ±最小关注效应之内,按等效性框架报告为「没有值得据此改变做法的差别」(Lakens, Scheel, and Isager 2018)

样本量

推导

# Chapter 8. Sample size for the pre-registered policy comparison. Design: a
# duplicate design in which the SAME deals are replayed with two decision
# policies, scored at the first decision at which the policies differ, against
# a fixed opponent pool. Primary
# outcome: the mean points difference per scored decision. The pairing is what
# makes the trial feasible (p11); the program sizes it on the paired standard
# deviation and then checks the normal-theory answer against a simulation with
# a realistic, heavy-tailed outcome distribution, because mahjong scores are
# not normal and the t test's behaviour in the tail matters.
import math
import random
from statistics import NormalDist

Z = NormalDist().inv_cdf
SESOI = 0.010           # buy-ins per scored decision
SD_DIFF = 0.120         # paired s.d. from the duplicate design (p11)

def n_paired(d=SESOI, sd=SD_DIFF, power=0.90, alpha=0.05):
    return (Z(1 - alpha / 2) + Z(power)) ** 2 * sd ** 2 / d ** 2

# a heavy-tailed paired difference: mostly zero, occasionally a large swing
def draw_diff(rng, delta):
    u = rng.random()
    if u < 0.70:
        return delta + rng.gauss(0.0, 0.02)          # small or no consequence
    if u < 0.94:
        return delta + rng.gauss(0.0, 0.12)          # a normal-sized swing
    # a big hand changes hands
    return delta + rng.gauss(0.0, 0.45)

def one_trial(rng, n, delta):
    xs = [draw_diff(rng, delta) for _ in range(n)]
    m = math.fsum(xs) / n
    v = math.fsum((x - m) ** 2 for x in xs) / (n - 1)
    se = math.sqrt(v / n)
    t = m / se if se > 0 else 0.0
    return t > Z(0.975), m

if __name__ == "__main__":
    print(
        "Duplicate design: the same deals replayed under two policies,"
        " scored at"
    )
    print("the first differing decision, against a fixed opponent pool.")
    print("SESOI {:.3f} buy-ins per scored decision; paired s.d. {:.3f}.\n"
          .format(SESOI, SD_DIFF))

    n0 = n_paired()
    print("normal theory: n = {:,.1f} -> {:,} scored decisions"
          .format(n0, math.ceil(n0)))

    rng = random.Random(5514)
    # empirical s.d. of the heavy-tailed generator, for comparison
    samp = [draw_diff(rng, 0.0) for _ in range(200000)]
    m = math.fsum(samp) / len(samp)
    sd_emp = math.sqrt(
        math.fsum((x - m) ** 2 for x in samp) / (len(samp) - 1)
    )
    print("generator's actual s.d.: {:.4f} (the planning value was {:.3f})"
          .format(sd_emp, SD_DIFF))
    n_adj = n_paired(sd=sd_emp)
    print("normal theory at the generator's s.d.: {:,} scored decisions"
          .format(math.ceil(n_adj)))

    print("\nsimulation with the heavy-tailed paired difference:")
    chosen = None
    for n in (math.ceil(n_adj), 12000, 15000, 18000):
        res = [one_trial(rng, n, SESOI) for _ in range(600)]
        pw = sum(r[0] for r in res) / len(res)
        se = math.sqrt(pw * (1 - pw) / len(res))
        print("   n = {:6,}: power {:.3f} (s.e. {:.3f})".format(n, pw, se))
        if chosen is None and pw - 2 * se >= 0.90:
            chosen = n
    if chosen is None:
        chosen = 18000
    t1 = sum(one_trial(rng, chosen, 0.0)[0] for _ in range(600)) / 600
    print("   smallest n with power - 2 s.e. >= 0.90: {:,}".format(chosen))
    print(
        "   one-sided error rate with no effect: {:.3f} (nominal 0.025)"
        .format(t1)
    )

    print(
        "\nplanned: {:,} scored decisions. At roughly one scored decision per"
        " hand".format(chosen)
    )
    print(
        "and 60 hands per session, that is about {:,.0f} sessions of play, or"
        .format(chosen / 60)
    )
    print("a few weeks of continuous self-play for a software agent.")

    print("\nsensitivity (normal theory):")
    print("   SESOI \\ paired s.d.      0.08      0.12      0.20")
    for d in (0.005, 0.010, 0.020):
        print("   {:.3f} buy-ins       ".format(d) + "".join(
            "{:10,.0f}".format(math.ceil(n_paired(d=d, sd=s)))
            for s in (0.08, 0.12, 0.20)))

    print(
        "\nTwo things this sizing makes explicit. First, a human cannot run"
        " this"
    )
    print(
        "trial: {:,} scored decisions is beyond what anyone plays"
        " deliberately, so".format(chosen)
    )
    print(
        "the comparison belongs to software agents replaying stored deals,"
        " and the"
    )
    print(
        "conclusions then apply to the policies rather than to people."
        " Second, the"
    )
    print(
        "SESOI of {:.3f} buy-ins per decision is small in absolute terms and"
        " still".format(SESOI)
    )
    print(
        "needs this many observations, which is the quantitative reason the"
        " strategy"
    )
    print(
        "debates in this game remain unsettled: the sample sizes required"
        " have"
    )
    print("rarely been assembled.")
# Duplicate design: the same deals replayed under two policies, scored at
# the first differing decision, against a fixed opponent pool.
# SESOI 0.010 buy-ins per scored decision; paired s.d. 0.120.
#
# normal theory: n = 1,513.1 -> 1,514 scored decisions
# generator's actual s.d.: 0.1256 (the planning value was 0.120)
# normal theory at the generator's s.d.: 1,657 scored decisions
#
# simulation with the heavy-tailed paired difference:
#    n =  1,657: power 0.880 (s.e. 0.013)
#    n = 12,000: power 1.000 (s.e. 0.000)
#    n = 15,000: power 1.000 (s.e. 0.000)
#    n = 18,000: power 1.000 (s.e. 0.000)
#    smallest n with power - 2 s.e. >= 0.90: 12,000
#    one-sided error rate with no effect: 0.033 (nominal 0.025)
#
# planned: 12,000 scored decisions. At roughly one scored decision per hand
# and 60 hands per session, that is about 200 sessions of play, or
# a few weeks of continuous self-play for a software agent.
#
# sensitivity (normal theory):
#    SESOI \ paired s.d.      0.08      0.12      0.20
#    0.005 buy-ins            2,690     6,053    16,812
#    0.010 buy-ins              673     1,514     4,203
#    0.020 buy-ins              169       379     1,051
#
# Two things this sizing makes explicit. First, a human cannot run this
# trial: 12,000 scored decisions is beyond what anyone plays deliberately, so
# the comparison belongs to software agents replaying stored deals, and the
# conclusions then apply to the policies rather than to people. Second, the
# SESOI of 0.010 buy-ins per decision is small in absolute terms and still
# needs this many observations, which is the quantitative reason the strategy
# debates in this game remain unsettled: the sample sizes required have
# rarely been assembled.

正态理论在配对标准差 0.120 下给出 1,514 个计分决策;而本书用一个重尾的配对差值生成器复核,因为这个博弈的点数不是正态的:大多数决策后果很小,偶尔一手大牌改变归属。模拟给出的答案大于正态理论的答案,按「把握度减两个标准误不低于 0.90」取 12,000 个计分决策。

按每手牌约一个计分决策、每场 60 手计,这约是 200 场,对软件代理是几周的自对局。两点因此写明:人类无法跑这个试验,所以这个比较属于软件代理,结论也就适用于策略而不适用于人;而 0.010 个买入/决策这样小的最小关注效应仍需这么多观测,正是这个博弈的策略争论长期不决的量化理由。

判定规则

推导

# Chapter 8. Decision rule, fixed before the data are examined. Primary
# comparison: policy B against policy A in the duplicate design. D = mean
# points difference per scored decision, in buy-ins, with its 95 % confidence
# interval. POSITIVE D favours B. SESOI 0.010 buy-ins per scored decision.
# Gates, checked first: fewer than the planned number of scored decisions; the
# opponent pool changed during the run; the two policies' scored decisions not
# drawn from the same deals (pairing broken). Pre-specified qualifiers: the
# share of scored decisions in which the policies agreed (if very high, the
# comparison concerns a rare situation and does not generalise), and the
# heavy-tail check (if a small number of hands dominate the total, the
# interval is unreliable and a trimmed or bootstrap interval is reported too).
SESOI = 0.010

def verdict(d, lo, hi, n_scored, n_planned=12000, pool_changed=False,
            pairing_broken=False, agree_share=0.5, top1_share=0.05):
    for bad, why in (
        (
            pairing_broken,
            (
                "pairing broken: the two policies were not"
                " scored on the same deals"
            ),
        ),
        (pool_changed, "the opponent pool changed during the run"),
        (
            n_scored < 0.9 * n_planned,
            "only {:,} of {:,} planned scored decisions".format(
                n_scored, n_planned
            ),
        ),
    ):
        if bad:
            return "D = {:+.4f}: NOT INTERPRETABLE -\n    {}".format(d, why)
    if lo >= SESOI:
        v = "B better by at least the SESOI"
    elif lo > 0:
        v = "B better, but not shown to reach the SESOI"
    elif hi < 0:
        v = "A better"
    elif lo >= -SESOI and hi <= SESOI:
        v = "no difference worth acting on"
    else:
        v = "unclear:\n    the interval is too wide to decide"
    notes = []
    if agree_share > 0.95:
        notes.append("the policies agreed on {:.0%} of decisions, so this"
                     "\n      compares a rare situation and does not describe"
                     " overall play"
                     .format(agree_share))
    if top1_share > 0.20:
        notes.append(
            "the largest 1 % of hands account for {:.0%} of the total"
            "\n      difference: report the trimmed and bootstrap"
            " intervals too".format(top1_share)
        )
    out = "D = {:+.4f} ({:+.4f} to {:+.4f}): {}".format(d, lo, hi, v)
    for nt in notes:
        out += "\n    qualifier: " + nt
    return out

CASES = [
    (0.0180, 0.0112, 0.0248, 12000),
    (0.0062, 0.0014, 0.0110, 12000),
    (0.0008, -0.0040, 0.0056, 12000),
    (-0.0140, -0.0205, -0.0075, 12000),
    (0.0090, -0.0180, 0.0360, 12000),
    (0.0150, 0.0082, 0.0218, 12000, 12000, False, False, 0.985),
    (0.0150, 0.0082, 0.0218, 12000, 12000, False, False, 0.5, 0.42),
    (0.0200, 0.0130, 0.0270, 8000),
    (0.0200, 0.0130, 0.0270, 12000, 12000, True),
    (0.0200, 0.0130, 0.0270, 12000, 12000, False, True),
]

if __name__ == "__main__":
    for c in CASES:
        print(verdict(*c))
    print("\nSecondary outcomes (win rate, deal-in rate, mean placement, the")
    print(
        "components of the score) are reported with intervals and never"
        " change the"
    )
    print(
        "verdict. No claim is made about equilibrium or unexploitability: per"
        " p08,"
    )
    print(
        "the result is a comparison against the FIXED opponent pool used,"
        " and a"
    )
    print("different pool is a different experiment.")
    print(
        "\nThe example numbers above are invented, to show what each branch"
        " prints."
    )
# D = +0.0180 (+0.0112 to +0.0248): B better by at least the SESOI
# D = +0.0062 (+0.0014 to +0.0110): B better, but not shown to reach the SESOI
# D = +0.0008 (-0.0040 to +0.0056): no difference worth acting on
# D = -0.0140 (-0.0205 to -0.0075): A better
# D = +0.0090 (-0.0180 to +0.0360): unclear:
#     the interval is too wide to decide
# D = +0.0150 (+0.0082 to +0.0218): B better, but not shown to reach the SESOI
#     qualifier: the policies agreed on 98% of decisions, so this
#       compares a rare situation and does not describe overall play
# D = +0.0150 (+0.0082 to +0.0218): B better, but not shown to reach the SESOI
#     qualifier: the largest 1 % of hands account for 42% of the total
#       difference: report the trimmed and bootstrap intervals too
# D = +0.0200: NOT INTERPRETABLE -
#     only 8,000 of 12,000 planned scored decisions
# D = +0.0200: NOT INTERPRETABLE -
#     the opponent pool changed during the run
# D = +0.0200: NOT INTERPRETABLE -
#     pairing broken: the two policies were not scored on the same deals
#
# Secondary outcomes (win rate, deal-in rate, mean placement, the
# components of the score) are reported with intervals and never change the
# verdict. No claim is made about equilibrium or unexploitability: per p08,
# the result is a comparison against the FIXED opponent pool used, and a
# different pool is a different experiment.
#
# The example numbers above are invented, to show what each branch prints.

前置条件有三项:计分决策数不足计划的九成、对手池在运行期间改变、配对被破坏(两种策略没有在同样的牌局上计分)。任何一项不满足,无论主要结局如何都写「不可解释」。两条限定语:两策略一致比例过高(则该比较只涉及罕见情形,不描述整体打法);少数牌局主导总差值(则另报修尾与自助法区间)。

这项比较不能说明什么

第一,它只比较两个策略,在一个固定对手池上,按一组固定规则。第二,它不作任何关于均衡或不可利用性的主张(第六章)。第三,它只在首次分歧的决策上计分,所以不回答「整手牌的打法」这类问题。第四,它不涉及人类玩家,所以不说明任何人类能否执行该策略。第五,若结果是「没有值得据此改变做法的差别」,那就是结论,不能用次要结局替代。

第四部分 关于钱的那一节

最后一节与前面不同:它不是策略,而是算术。

推导

# Chapter 8. The arithmetic of playing for money, written out because a book
# of strategy is incomplete without it. Everything in Chapters 1 to 7 is about
# the game as a game. This programme answers a different question: for a
# player with a given edge, fee and stake, what does the distribution of
# outcomes look like over a year, and what fraction of otherwise identical
# players end the year down? The answer is used in Chapter 8 to state plainly
# what the book does and does not support, and it is the reason the book
# recommends studying the game rather than financing the study by playing it.
import math
import random
from statistics import NormalDist

N = NormalDist()
OUTCOMES = [(+0.90, 0.155), (+0.30, 0.110), (-0.10, 0.450), (-0.35, 0.225),
            (-0.60, 0.060)]

def with_fee(fee):
    return [(x - fee, p) for x, p in OUTCOMES]

def mean(d):
    return math.fsum(x * p for x, p in d)

def sd(d):
    m = mean(d)
    return math.sqrt(math.fsum(p * (x - m) ** 2 for x, p in d))

def year_stats(hands, stake, fee):
    d = with_fee(fee)
    mu = mean(d) * stake * hands
    s = sd(d) * stake * math.sqrt(hands)
    return mu, s, N.cdf(-mu / s) if s > 0 else (0.0 if mu > 0 else 1.0)

if __name__ == "__main__":
    HANDS = 60 * 52                 # one session a week for a year
    print(
        "One session of 60 hands a week for a year: {:,} hands.".format(HANDS)
    )
    print("Stake 1 buy-in per hand throughout.\n")

    print(
        "A. A skilled player, before fees, has the edge of the p09"
        " distribution:"
    )
    print(
        "   fee per hand   expected year   s.d. of the year   P(down over the"
        " year)"
    )
    for fee in (0.000, 0.010, 0.020, 0.040, 0.073):
        mu, s, p = year_stats(HANDS, 1.0, fee)
        print("   {:12.3f} {:15.1f} {:18.1f} {:23.3f}".format(fee, mu, s, p))

    print(
        "\n   Note the last column at a fee that still leaves a positive"
        " edge. At a"
    )
    mu, s, p = year_stats(HANDS, 1.0, 0.010)
    print(
        "   fee of 0.010 the expected year is {:+.1f} buy-ins and yet"
        " {:.0%} of".format(mu, p)
    )
    print(
        "   players with exactly this edge finish the year down. A positive"
    )
    print(
        "   expectation does not make a year of play likely to be"
        " profitable; it"
    )
    print(
        "   makes it profitable on average over far more hands than a year"
        " holds."
    )

    print(
        "\nB. How many hands before the probability of being down falls below"
        " 5 %?"
    )
    for fee in (0.000, 0.010, 0.020, 0.040):
        d = with_fee(fee)
        mu1, s1 = mean(d), sd(d)
        if mu1 <= 0:
            print("   fee {:.3f}: never (no edge)".format(fee))
            continue
        # need mu1 n / (s1 sqrt(n)) >= 1.645
        n_need = (1.645 * s1 / mu1) ** 2
        print("   fee {:.3f}: {:,.0f} hands ({:,.0f} years at this rate)"
              .format(fee, math.ceil(n_need), n_need / HANDS))

    print(
        "\n   At a fee of 0.02 a player needs on the order of a decade of"
        " weekly"
    )
    print(
        "   play before a profit is more likely than not by this standard,"
        " and that"
    )
    print(
        "   is for a player whose edge is as large as the p09 distribution"
        " assumes"
    )
    print(
        "   and who does not get worse, tired or unlucky in their choice of"
        " table."
    )

    print(
        "\nC. And for a player with NO edge, which is the position of most"
        " people:"
    )
    for fee in (0.000, 0.010, 0.020):
        d = [(x - fee, p) for x, p in OUTCOMES]
        d = [(x - mean(d), p) for x, p in d]     # remove the edge exactly
        d = [(x - fee, p) for x, p in d]
        mu, s = mean(d) * HANDS, sd(d) * math.sqrt(HANDS)
        print("   fee {:.3f}: expected year {:+.1f} buy-ins, s.d. {:.1f},"
              " P(down) {:.3f}".format(fee, mu, s, N.cdf(-mu / s)))
    print(
        "\n   With no edge and any fee at all, the expected year is negative"
        " and the"
    )
    print(
        "   probability of being down exceeds a half, by construction. There"
        " is no"
    )
    print(
        "   staking rule, bankroll or discipline that changes this:"
        " Chapter 6's"
    )
    print(
        "   Kelly result is explicitly zero stake when the edge is not"
        " positive."
    )

    print(
        "\nD. What the book therefore says. The models in Chapters 1 to 7 are"
        " about"
    )
    print(
        "   a game of imperfect information, and they are worth studying as"
        " that."
    )
    print(
        "   They do not establish that any reader has an edge, and"
        " programme p11"
    )
    print(
        "   showed that establishing one takes more hands than most players"
        " will"
    )
    print(
        "   ever record. Playing for money on the strength of an unmeasured"
        " edge is"
    )
    print(
        "   the error this programme quantifies, and the risk is not"
        " symmetric: the"
    )
    print("   losses are real and the edge is hypothetical.")
    print(
        "\n   Gambling can cause serious financial and personal harm, and"
        " it can"
    )
    print(
        "   become compulsive. Anyone who finds that they are playing more"
        " than"
    )
    print(
        "   they intended, staking more than they can afford, or chasing"
        " losses"
    )
    print(
        "   should treat that as the signal it is and seek help from a"
        " qualified"
    )
    print("   service in their own country. This book is a study of a game's")
    print(
        "   mathematics; it is not an inducement to bet on it, and the author"
    )
    print("   recommends studying the game without staking money on it.")
# One session of 60 hands a week for a year: 3,120 hands.
# Stake 1 buy-in per hand throughout.
#
# A. A skilled player, before fees, has the edge of the p09 distribution:
#    fee per hand   expected year   s.d. of the year   P(down over the year)
#           0.000            39.8               24.3                   0.051
#           0.010             8.6               24.3                   0.362
#           0.020           -22.6               24.3                   0.824
#           0.040           -85.0               24.3                   1.000
#           0.073          -188.0               24.3                   1.000
#
#    Note the last column at a fee that still leaves a positive edge. At a
#    fee of 0.010 the expected year is +8.6 buy-ins and yet 36% of
#    players with exactly this edge finish the year down. A positive
#    expectation does not make a year of play likely to be profitable; it
#    makes it profitable on average over far more hands than a year holds.
#
# B. How many hands before the probability of being down falls below 5 %?
#    fee 0.000: 3,146 hands (1 years at this rate)
#    fee 0.010: 67,611 hands (22 years at this rate)
#    fee 0.020: never (no edge)
#    fee 0.040: never (no edge)
#
#    At a fee of 0.02 a player needs on the order of a decade of weekly
#    play before a profit is more likely than not by this standard, and that
#    is for a player whose edge is as large as the p09 distribution assumes
#    and who does not get worse, tired or unlucky in their choice of table.
#
# C. And for a player with NO edge, which is the position of most people:
#    fee 0.000: expected year -0.0 buy-ins, s.d. 24.3, P(down) 0.500
#    fee 0.010: expected year -31.2 buy-ins, s.d. 24.3, P(down) 0.901
#    fee 0.020: expected year -62.4 buy-ins, s.d. 24.3, P(down) 0.995
#
#    With no edge and any fee at all, the expected year is negative and the
#    probability of being down exceeds a half, by construction. There is no
#    staking rule, bankroll or discipline that changes this: Chapter 6's
#    Kelly result is explicitly zero stake when the edge is not positive.
#
# D. What the book therefore says. The models in Chapters 1 to 7 are about
#    a game of imperfect information, and they are worth studying as that.
#    They do not establish that any reader has an edge, and programme p11
#    showed that establishing one takes more hands than most players will
#    ever record. Playing for money on the strength of an unmeasured edge is
#    the error this programme quantifies, and the risk is not symmetric: the
#    losses are real and the edge is hypothetical.
#
#    Gambling can cause serious financial and personal harm, and it can
#    become compulsive. Anyone who finds that they are playing more than
#    they intended, staking more than they can afford, or chasing losses
#    should treat that as the signal it is and seek help from a qualified
#    service in their own country. This book is a study of a game's
#    mathematics; it is not an inducement to bet on it, and the author
#    recommends studying the game without staking money on it.

A 段的要点在最后一列。在抽水 0.010、期望年度收益 +8.6+8.6 个买入时,仍有 36% 的玩家年终亏损。正期望不使一年的游戏可能获利;它使游戏在远多于一年的手数上平均获利。

B 段给出「多久才多半不亏」:抽水 0.010 时需要的手数以年计不止一年,而那是对一个优势像本书分布那样大、并且不会变差、不会疲劳、不会选错桌的玩家而言的。

C 段给出大多数人的处境:没有优势时,任何抽水都使期望为负、亏损概率超过一半。第六章的凯利结果在这里是明确的——优势不为正时的最优注码是零。

D 段是本书的立场。第一到七章的模型是关于一个不完全信息博弈的,它们作为那个是值得研究的。它们不证明任何读者有优势,而程序 p11 说明证明一个优势需要的手数多于大多数玩家会记录的手数。以未经测量的优势投注金钱,就是这一节量化的那个错误,而风险不对称:亏损是真实的,优势是假设的。

注

赌博相关危害。 赌博可能造成严重的财务与个人伤害,也可能成为强迫行为;问题赌博的流行率与相关因素有系统综述 (Calado and Griffiths 2016),而赌博障碍作为一个诊断实体有专门的综述 (Potenza et al. 2019)。任何人若发现自己玩得比预期多、下注超过能承受的范围、或在追赶损失,应把这当作信号,并向本国有资质的服务寻求帮助。本书的建议是:研究这个博弈,而不在它上面押钱。

利益披露

作者以 5D Library 的著作(包括本书)作为其社群课程、升维牌卡(5D Ascension Cards,Dimensional Shift Card Decks)与 Yafu AI 知识库的方法论基础,因此与本书主题有商业利益关系。本书不推销任何课程、牌卡、软件或服务,不提供任何博彩平台、代理或资金服务的链接或推荐,也不接受任何此类机构的赞助。本书不是赌博指南:它不建议任何人以金钱投注本书讨论的博弈,第八章最后一节用计算说明为什么以未经测量的优势投注是一个可量化的错误,并指出赌博可能造成严重的财务与个人伤害、也可能成为强迫行为。第八章的试验为软件代理之间的策略比较,由独立于作者企业的研究人员实施和分析,不涉及真实金钱投注,也不从作者的社群、学员或客户中招募;书中所有代码与研究方案均公开,供任何人检验。

本章小结

四步核查表从量的类别、方向、对手与样本四个方面阅读策略说法;用于通行推理时,它显示两类量被混用、至少三处方向相反、对手池多半不写明、而边际决策上的自信意见不被手数支持。样本量是本章最不舒服的部分:每手 0.01 个买入的改善需要每组约 39,766 手,而 10,000 手只能检出 0.0199 以上的改善;重复对局设计把所需手数降低约 53 倍,代价是只能在首次分歧的决策上计分。预注册的比较以策略 B 对策略 A 为唯一主要比较、每个计分决策的平均点数差为唯一主要结局,最小关注效应 0.010 个买入,需 12,000 个计分决策,判定规则写成代码,并把两策略一致比例与少数牌局主导总差列为限定条件;该比较不作任何关于均衡的主张。最后一节算出:抽水 0.010 时期望年度收益 +8.6+8.6 个买入而仍有 36% 的玩家年终亏损,没有优势时任何抽水都使期望为负,而优势不为正时的最优注码是零。

参考文献

Calado, Filipa, and Mark D. Griffiths. 2016. “Problem Gambling Worldwide: An Update and Systematic Review of Empirical Research (2000–2015).” Journal of Behavioral Addictions 5 (4): 592–613. https://doi.org/10.1556/2006.5.2016.073.
Lakens, Daniël, Anne M. Scheel, and Peder M. Isager. 2018. “Equivalence Testing for Psychological Research: A Tutorial.” Advances in Methods and Practices in Psychological Science 1 (2): 259–69. https://doi.org/10.1177/2515245918770963.
Nosek, Brian A., Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor. 2018. “The Preregistration Revolution.” Proceedings of the National Academy of Sciences 115 (11): 2600–2606. https://doi.org/10.1073/pnas.1708274114.
Potenza, Marc N., Iris M. Balodis, Jeffrey Derevensky, Jon E. Grant, Nancy M. Petry, Antonio Verdejo-Garcia, and Sarah W. Yip. 2019. “Gambling Disorder.” Nature Reviews Disease Primers 5: 51. https://doi.org/10.1038/s41572-019-0099-7.
Schulz, Kenneth F., Douglas G. Altman, and David Moher. 2010. “CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomised Trials.” BMJ 340: c332. https://doi.org/10.1136/bmj.c332.
Wilson, Greg, D. A. Aruliah, C. Titus Brown, Neil P. Chue Hong, Matt Davis, Richard T. Guy, Steven H. D. Haddock, et al. 2014. “Best Practices for Scientific Computing.” PLOS Biology 12 (1): e1001745. https://doi.org/10.1371/journal.pbio.1001745.

Fan, Y. (2026). Imperfect-Information Stochastic Games, Bayesian Hand-Reading and Counterfactual Regret Minimisation in Four-Player Tile Dynamics: From Markov Transitions of the Tile Pool and the Optimal Stopping of Defensive Folding to Kelly Allocation in a Multiplayer Game, with a Textual Examination of the Orthodox and the Unorthodox in the Classical Military Writers [四人不完全信息随机博弈、贝叶斯读牌与反事实遗憾最小化] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24020464 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).