5D Library › 演化博弈、直接与间接互惠及合作涌现的数学模型 · English

第三章 直接互惠:锦标赛、噪声与宽容

两次锦标赛

特里弗斯 1971 年提出,无亲缘关系的个体之间可以通过「互惠利他」维持合作:今天我帮你,明天你帮我;只要将来还会相遇,欺骗者就会失去将来的帮助 (Trivers 1971)。这个想法在博弈论中的形式是重复囚徒困境。

证据

阿克塞尔罗德锦标赛。 阿克塞尔罗德邀请博弈论与相关学科的研究者提交在重复囚徒困境中下棋的计算机程序,让它们两两对局(每个程序也与自己的副本对局),以总分排名。第一次锦标赛有 14 个程序(另加一个随机策略),每局 200 轮;获胜的是拉波波特提交的一报还一报(TFT):第一轮合作,以后每轮重复对方上一轮的选择 (Axelrod 1980a)。第一次的结果与分析公布之后,第二次锦标赛收到 62 个程序,对局长度改为随机,一报还一报再次获胜 (Axelrod 1980b)。阿克塞尔罗德把成功程序的特点概括为:不首先背叛(善良),对背叛立即报复(可激怒),报复之后回到合作(宽容),行为简单清楚。阿克塞尔罗德与生物学家汉密尔顿 1981 年在《科学》上把这些结果与生物演化联系起来 (Axelrod and Hamilton 1981)。

有两点常被忽略。第一,一报还一报在任何一局中都不可能比对手得分更多:它只在对方背叛之后背叛,所以它「占对方便宜」的次数不会超过被对方占便宜的次数。它的总分高,是因为它能与许多程序维持合作,而不是因为它击败了谁。第二,名次取决于参赛者的构成。阿克塞尔罗德本人也通过改变参赛构成的「假设锦标赛」检验了结论的稳健性 (Axelrod 1980b)。

推导

# A small round-robin tournament in the spirit of Axelrod (1980): eleven simple rules;
# every pair (and each rule against its own twin) plays 200 rounds of the prisoner's
# dilemma with Axelrod's payoffs R = 3, S = 0, T = 5, P = 1. With execution noise eps,
# each intended move is flipped with probability eps. Scores are average points per
# round. This is NOT a replay of Axelrod's tournaments, whose 14 and 62 entries were
# more varied; rankings depend on the field of entries.
import random
rng = random.Random(5)
PAY = {("C", "C"): 3, ("C", "D"): 0, ("D", "C"): 5, ("D", "D"): 1}

def allc(m, o): return "C"
def alld(m, o): return "D"
def tft(m, o): return o[-1] if o else "C"
def stft(m, o): return o[-1] if o else "D"                 # suspicious TFT
def tf2t(m, o): return "D" if o[-2:] == ["D", "D"] else "C"
def grim(m, o): return "D" if "D" in o else "C"
def wsls(m, o):                                            # win-stay, lose-shift (Pavlov)
    if not m: return "C"
    return m[-1] if PAY[(m[-1], o[-1])] >= 3 else ("D" if m[-1] == "C" else "C")
def gtft(m, o):                                            # forgives a D with prob. 1/3
    if not o or o[-1] == "C": return "C"
    return "C" if rng.random() < 1 / 3 else "D"
def joss(m, o):                                            # TFT, but sneaks in 10 % D
    if o and o[-1] == "D": return "D"
    return "D" if rng.random() < 0.1 else "C"
def tester(m, o):              # probes with D in round 1; if the other side does not hit
    if not m: return "D"       # back in round 2, it defects every other round; else TFT
    if len(m) == 1: return "C"
    if o[1] == "D": return o[-1]
    return "D" if len(m) % 2 == 0 else "C"
def rand(m, o): return "C" if rng.random() < 0.5 else "D"
RULES = {"TFT": tft, "TF2T": tf2t, "GTFT": gtft, "WSLS": wsls, "GRIM": grim, "ALLC": allc,
         "STFT": stft, "JOSS": joss, "RANDOM": rand, "ALLD": alld,
         "TESTER": tester}

def match(f, g, eps, n=200):
    a, b, sa, sb = [], [], 0, 0
    for _ in range(n):
        x, y = f(a, b), g(b, a)
        if rng.random() < eps: x = "D" if x == "C" else "C"
        if rng.random() < eps: y = "D" if y == "C" else "C"
        a.append(x); b.append(y); sa += PAY[(x, y)]; sb += PAY[(y, x)]
    return sa / n, sb / n

def tournament(eps, reps):
    tot = {k: 0.0 for k in RULES}
    names = list(RULES)
    for _ in range(reps):
        for i, p in enumerate(names):
            for q in names[i:]:
                s1, s2 = match(RULES[p], RULES[q], eps)
                if p == q: tot[p] += s1
                else: tot[p] += s1; tot[q] += s2
    return {k: v / (reps * len(names)) for k, v in tot.items()}

for eps, reps in ((0.0, 20), (0.01, 20), (0.05, 20)):
    res = sorted(tournament(eps, reps).items(), key=lambda kv: -kv[1])
    print(f"noise {eps:4.2f}: " + "  ".join(f"{k} {v:.2f}" for k, v in res[:5]))
    print("            " + "  ".join(f"{k} {v:.2f}" for k, v in res[5:]))

print("a rule against its own twin, points per round (mean of 200 matches)")
print("        noise 0   1 %, 200 rounds   1 %, 5000 rounds   5 %, 5000 rounds")
for name in ("TFT", "GTFT", "WSLS", "GRIM"):
    row = []
    for eps, n in ((0.0, 200), (0.01, 200), (0.01, 5000), (0.05, 5000)):
        f = RULES[name]
        row.append(sum(match(f, f, eps, n)[0] for _ in range(200)) / 200)
    print(f"  {name:5s}  {row[0]:.2f}        {row[1]:.2f}              {row[2]:.2f}"
          f"               {row[3]:.2f}")
# noise 0.00: GTFT 2.64  TF2T 2.54  TFT 2.50  WSLS 2.43  ALLC 2.43
#             TESTER 2.31  GRIM 2.29  RANDOM 2.21  STFT 2.04  ALLD 1.86  JOSS 1.86
# noise 0.01: GTFT 2.43  TF2T 2.37  TESTER 2.31  WSLS 2.30  TFT 2.25
#             RANDOM 2.20  ALLC 2.16  STFT 2.13  GRIM 2.07  JOSS 1.98  ALLD 1.89
# noise 0.05: TESTER 2.31  GTFT 2.30  TF2T 2.25  WSLS 2.25  RANDOM 2.23
#             TFT 2.21  STFT 2.18  JOSS 2.15  GRIM 2.04  ALLC 2.04  ALLD 2.03
# a rule against its own twin, points per round (mean of 200 matches)
#         noise 0   1 %, 200 rounds   1 %, 5000 rounds   5 %, 5000 rounds
#   TFT    3.00        2.43              2.26               2.25
#   GTFT   3.00        2.97              2.97               2.86
#   WSLS   3.00        2.95              2.95               2.78
#   GRIM   3.00        1.51              1.05               1.15

程序用 11 个简单规则做一次小型的循环赛,收益采用阿克塞尔罗德的数值:双方合作各得 R=3R = 3,双方背叛各得 P=1P = 1,背叛合作者得 T=5T = 5,被背叛得 S=0S = 0。这不是阿克塞尔罗德锦标赛的重演。在这个参赛构成中,没有噪声时一报还一报排第三(2.50),排在慷慨一报还一报(GTFT,被背叛后以 1/3 的概率仍然合作,2.64)与「一报还两报」(TF2T,对方连续两次背叛才报复,2.54)之后。有 1% 的执行错误时,一报还一报降到第五(2.25);5% 时降到第六。名次随噪声与对手而变,这正是上面第二点的意思。

程序的第二部分更清楚。没有噪声时,每个规则与自己的副本都得到全合作的 3.00。1% 的执行错误下,两个一报还一报在 200 轮中平均只得 2.43,在 5,000 轮中为 2.26;冷酷触发(GRIM,一次背叛之后永远背叛)更糟,200 轮中为 1.51。原因是:一方偶然背叛一次,另一方下一轮报复,前者又报复回来,两者陷入「你一次、我一次」的交替背叛,直到下一个错误把它们带回合作或带入相互背叛。慷慨一报还一报与赢定输移(WSLS)在同样条件下不低于 2.95。

噪声下的马尔可夫链

把「一报还一报陷入报复循环」说精确,需要一个工具。只看上一轮结果的策略(记忆一步的策略)可以写成四个数 𝒑=(pCC,pCD,pDC,pDD)\mathbf p = (p_{CC}, p_{CD}, p_{DC}, p_{DD}):在上一轮自己与对方分别为 CC、CD、DC、DD 之后,这一轮合作的概率。一报还一报是 (1,0,1,0)(1, 0, 1, 0),赢定输移是 (1,0,0,1)(1, 0, 0, 1):上一轮得到 RR 或 TT(「赢」)就保持原来的选择,得到 SS 或 PP(「输」)就换一个。两个这样的策略对局,上一轮的结果构成一个四状态的马尔可夫链;执行错误 ε\varepsilon 使每个概率变为 (1−ε)p+ε(1−p)(1 – \varepsilon)p + \varepsilon(1 – p)。长期平均收益由链的平稳分布给出。

推导

# Memory-one strategies p = (p_CC, p_CD, p_DC, p_DD): probability of cooperating after
# (own last move, other's last move). With execution error eps an intended move is
# flipped. Two such players form a 4-state Markov chain; long-run payoffs come from its
# stationary distribution (infinitely repeated game, no discounting).
import itertools

def flip(p, eps): return tuple((1 - eps) * x + eps * (1 - x) for x in p)

def stationary(p, q, eps):
    p, q = flip(p, eps), flip(q, eps)
    qs = (q[0], q[2], q[1], q[3])          # the other player sees CD as DC and vice versa
    M = [[a * b, a * (1 - b), (1 - a) * b, (1 - a) * (1 - b)] for a, b in zip(p, qs)]
    A = [[M[j][i] - (i == j) for j in range(4)] for i in range(4)]   # v (M - I) = 0
    A[3] = [1.0] * 4                                                  # and sum(v) = 1
    rhs = [0.0, 0.0, 0.0, 1.0]
    for col in range(4):                     # Gaussian elimination with pivoting
        piv = max(range(col, 4), key=lambda r: abs(A[r][col]))
        A[col], A[piv] = A[piv], A[col]; rhs[col], rhs[piv] = rhs[piv], rhs[col]
        for r in range(4):
            if r != col and A[r][col]:
                f = A[r][col] / A[col][col]
                A[r] = [x - f * y for x, y in zip(A[r], A[col])]; rhs[r] -= f * rhs[col]
    return [rhs[i] / A[i][i] for i in range(4)]

def payoffs(p, q, eps, R, S, T, P):
    v = stationary(p, q, eps)
    return (v[0] * R + v[1] * S + v[2] * T + v[3] * P,
            v[0] * R + v[1] * T + v[2] * S + v[3] * P)

b, c, eps = 3.0, 1.0, 0.01
DON = (b - c, -c, b, 0.0)                    # donation game: R, S, T, P
g = 1 - c / b
ST = {"TFT": (1, 0, 1, 0), "GTFT": (1, g, 1, g), "GTFT 0.5": (1, 0.5, 1, 0.5),
      "WSLS": (1, 0, 0, 1),
      "ALLC": (1, 1, 1, 1), "ALLD": (0, 0, 0, 0)}
print(f"donation game b = {b:.0f}, c = {c:.0f}, error {eps:.0%};"
      f" GTFT forgives with 1 - c/b = {g:.3f}")
print("resident   payoff vs itself   ALLD invader gets   ALLC invader gets")
for r in ("TFT", "GTFT", "GTFT 0.5", "WSLS"):
    own = payoffs(ST[r], ST[r], eps, *DON)[0]
    vd = payoffs(ST["ALLD"], ST[r], eps, *DON)[0]
    vc = payoffs(ST["ALLC"], ST[r], eps, *DON)[0]
    print(f"  {r:8s}     {own:.3f}             {vd:.3f}              {vc:.3f}")
v = stationary(ST["TFT"], ST["TFT"], eps)
print("TFT pair, long-run share of CC, CD, DC, DD: " + ", ".join(f"{x:.3f}" for x in v))

# How much generosity can GTFT afford? Scan q; GTFT(q) is a best reply to itself if no
# deterministic memory-one rule (16 of them) earns more against it. Compare with
# q* = min{1 - (T - R)/(R - S), (R - P)/(T - P)} (Nowak & Sigmund 1992; Molander 1985).
RULE16 = list(itertools.product((0, 1), repeat=4))
def stable_range(R, S, T, P, e=1e-5):
    ok = []
    for k in range(1, 1000):
        q = k / 1000; gt = (1, q, 1, q)
        own = payoffs(gt, gt, e, R, S, T, P)[0]
        if max(payoffs(r, gt, e, R, S, T, P)[0] for r in RULE16) <= own + 1e-3:
            ok.append(q)
    return min(ok), max(ok)
for name, (R, S, T, P) in (("donation b=3, c=1", DON), ("Axelrod 3,0,5,1", (3, 0, 5, 1)),
                           ("R,S,T,P = 3,0,4,1", (3, 0, 4, 1))):
    f1, f2 = 1 - (T - R) / (R - S), (R - P) / (T - P)
    lo, hi = stable_range(R, S, T, P)
    print(f"{name:18s}: stable for q in [{lo:.3f}, {hi:.3f}];"
          f" formula min({f1:.3f}, {f2:.3f}) = {min(f1, f2):.3f}")

# WSLS against ALLD: WSLS alternates C, D, so ALLD earns (T + P)/2 = b/2 per round.
print("WSLS resident, error 1 %: own payoff vs ALLD invader's payoff")
for bb in (1.5, 2.0, 2.5, 4.0):
    D = (bb - 1, -1, bb, 0)
    print(f"  b/c = {bb}: {payoffs(ST['WSLS'], ST['WSLS'], eps, *D)[0]:.3f} vs"
          f" {payoffs(ST['ALLD'], ST['WSLS'], eps, *D)[0]:.3f}")

# Press & Dyson (2012) extortion with chi = 3, Axelrod payoffs, phi = 1/26: p = (1 - 2
# phi (chi - 1), 1 - phi (1 + 4 chi), phi (4 + chi), 0) = (11/13, 1/2, 7/26, 0).
chi, phi = 3, 1 / 26
EXT = (1 - 2 * phi * (chi - 1), 1 - phi * (1 + 4 * chi), phi * (4 + chi), 0)
print("extortioner (chi = 3) vs opponent: (s_X - P)/(s_Y - P), s_X, s_Y")
for name, q in (("ALLC", (1, 1, 1, 1)), ("TFT", (1, 0, 1, 0)), ("WSLS", (1, 0, 0, 1)),
                ("random 0.5", (0.5,) * 4), ("GTFT 1/3", (1, 1 / 3, 1, 1 / 3))):
    sx, sy = payoffs(EXT, q, 0.0, 3, 0, 5, 1)
    ratio = f"{(sx - 1) / (sy - 1):.3f}" if abs(sy - 1) > 1e-9 else "0 = 3 x 0"
    print(f"  {name:10s}: {ratio:9s}   {sx:.3f}  {sy:.3f}")
sx, sy = payoffs(EXT, EXT, 0.0, 3, 0, 5, 1)
print(f"  two extortioners: {sx:.3f} each; two cooperators would get 3.000")
# donation game b = 3, c = 1, error 1%; GTFT forgives with 1 - c/b = 0.667
# resident   payoff vs itself   ALLD invader gets   ALLC invader gets
#   TFT          1.000             0.049              1.951
#   GTFT         1.970             1.990              1.970
#   GTFT 0.5     1.961             1.505              1.965
#   WSLS         1.941             1.490              0.510
# TFT pair, long-run share of CC, CD, DC, DD: 0.250, 0.250, 0.250, 0.250
# donation b=3, c=1 : stable for q in [0.020, 0.666]; formula min(0.667, 0.667) = 0.667
# Axelrod 3,0,5,1   : stable for q in [0.010, 0.333]; formula min(0.333, 0.500) = 0.333
# R,S,T,P = 3,0,4,1 : stable for q in [0.020, 0.666]; formula min(0.667, 0.667) = 0.667
# WSLS resident, error 1 %: own payoff vs ALLD invader's payoff
#   b/c = 1.5: 0.485 vs 0.740
#   b/c = 2.0: 0.971 vs 0.990
#   b/c = 2.5: 1.456 vs 1.240
#   b/c = 4.0: 2.912 vs 1.990
# extortioner (chi = 3) vs opponent: (s_X - P)/(s_Y - P), s_X, s_Y
#   ALLC      : 3.000       3.727  1.909
#   TFT       : 0 = 3 x 0   1.000  1.000
#   WSLS      : 3.000       2.364  1.455
#   random 0.5: 3.000       2.562  1.521
#   GTFT 1/3  : 3.000       2.616  1.539
#   two extortioners: 1.000 each; two cooperators would get 3.000

结果有四点。

第一,两个一报还一报在 1% 的错误下,长期停留在 CC、CD、DC、DD 四个状态的时间各占 25%,平均收益 (R+S+T+P)/4(R + S + T + P)/4,在 b=3b = 3、c=1c = 1 的捐献博弈中为 1.000,只有全合作(2)的一半。这个结论与错误率无关:错误率越低,走到这个平稳状态所需的时间越长,但只要错误率大于零,长期结果都一样(图二左)。它还意味着一个更隐蔽的问题:在一报还一报的种群中,永远合作者(ALLC)反而得到 1.951,高于一报还一报自己的 1.000,所以一报还一报的种群会被「无条件的好人」逐渐稀释,为永远背叛者的入侵铺路。

第二,慷慨一报还一报的原谅概率。诺瓦克与西格蒙德 1992 年在一个含有许多随机策略的种群模拟中发现,一报还一报扮演的是「催化剂」:它先把背叛者清除,然后被比它更宽容的慷慨一报还一报取代 (Nowak and Sigmund 1992)。慷慨一报还一报被背叛后仍以概率 qq 合作,qq 的上限为

q*=min⁡{1−T−RR−S,R−PT−P}q^* = \min\left\{1 – \frac{T – R}{R – S},\ \frac{R – P}{T – P}\right\}

这一结果此前已由莫兰德 1985 年在讨论「在自私而不确定的环境中最优的慷慨程度」时得到 (Molander 1985)。在捐献博弈中两项都等于 1−c/b1 – c/b。程序在所有 16 种确定性的记忆一步策略中寻找对慷慨一报还一报的最佳回应,发现它在 qq 不超过 0.666 时是对自身的最佳回应,与 1−c/b=0.6671 – c/b = 0.667 一致;在阿克塞尔罗德的收益下上限为 1/3,在 (R,S,T,P)=(3,0,4,1)(R, S, T, P) = (3, 0, 4, 1) 下为 2/3,都与公式吻合。需要注意,1−c/b1 – c/b 是能抵御永远背叛者的上限。莫兰德所说的「最优慷慨程度」,以及诺瓦克与西格蒙德模拟中演化出的慷慨一报还一报(在阿克塞尔罗德的收益下原谅概率约 1/3),指的都是这个边界上或附近的值:在不被背叛者占便宜的前提下尽可能宽容。但恰好取这个值时没有任何余量:没有错误时,永远背叛者每轮从它的原谅中得到 qb=b−cqb = b – c,与两个慷慨一报还一报相互合作的收益相同;有 1% 的错误时,永远背叛者对它的收益(1.990)已经略高于它自己的收益(1.970)。取 q=0.5q = 0.5 时,背叛者只得 1.505,而慷慨一报还一报自己得 1.961。框架中「原谅概率为 1−c/b1 – c/b」应读作「原谅概率不能超过 1−c/b1 – c/b,实际取值宜略低于它」。

第三,赢定输移的条件。诺瓦克与西格蒙德 1993 年发现,在有错误的种群模拟中,赢定输移常常胜过一报还一报:它能在一次错误之后两轮内恢复合作(双方都背叛一次,都「输」,于是一起换回合作),并且会利用无条件合作者 (Nowak and Sigmund 1993)。这个策略此前已由克雷恩夫妇 1989 年以「巴甫洛夫」之名研究 (Kraines and Kraines 1989)。但它的优势有条件:遇到永远背叛者时,赢定输移交替合作与背叛,背叛者每两轮得到一次 TT 与一次 PP,平均 (T+P)/2(T + P)/2。赢定输移的种群要抵御背叛者,需要 R>(T+P)/2R > (T + P)/2;在捐献博弈中即 b−c>b/2b – c > b/2,b/c>2b/c > 2。程序显示,1% 错误下 b/c=1.5b/c = 1.5 与 2.0 时背叛者的收益(0.740、0.990)高于赢定输移自己(0.485、0.971),b/c=2.5b/c = 2.5 与 4 时低于它。在阿克塞尔罗德的收益下,R=3R = 3 恰好等于 (T+P)/2=3(T + P)/2 = 3,赢定输移处在边界上。

第四,零行列式策略。普雷斯与戴森 2012 年发现,记忆一步策略中有一类能单方面在自己与对手的长期收益之间强加一个线性关系;其中的「勒索」策略使 sX−P=χ(sY−P)s_X – P = \chi\,(s_Y – P),χ>1\chi > 1,无论对手怎样做 (Press and Dyson 2012)。程序取 χ=3\chi = 3,对永远合作、赢定输移、随机策略与慷慨一报还一报,比值都恰好是 3.000;对一报还一报,双方都停在 PP,关系以 0=3×00 = 3 \times 0 的形式成立。勒索者在每一局中都不吃亏,但两个勒索者相遇只得 1.000,远低于两个合作者的 3.000。

演化选择哪一种

一局的输赢与种群中的演化是两回事。下面的程序把这些策略放进一个有限种群:每次一个随机个体以一定概率模仿另一个收益更高的个体,突变很少发生,于是种群在大部分时间里只有一种策略,偶尔被新策略取代。计算所有两两之间的固定概率,就能得到长期中种群停留在每种策略上的时间比例 (Imhof et al. 2007)。

推导

# Which memory-one rule does evolution pick? Finite population of N players, donation
# game with error eps = 1 %, long-run payoffs from p06's Markov chain. Imitation by
# pairwise comparison: a random player copies a random other with probability 1 / (1 +
# exp(-beta * (payoff difference))). With rare mutations (Fudenberg-Imhof limit, as in
# Imhof, Fudenberg & Nowak 2007) the population is almost always monomorphic; we compute
# all fixation probabilities and the long-run share of time spent in each strategy.
import importlib.util, io, contextlib, math, pathlib
spec = importlib.util.spec_from_file_location(
    "p06", pathlib.Path(__file__).resolve().parent / "p06_markov_noise.py")
p06 = importlib.util.module_from_spec(spec)
with contextlib.redirect_stdout(io.StringIO()):
    spec.loader.exec_module(p06)

def fixation(A, i, j, N, beta):
    """Probability that one i-mutant takes over a population of j-players."""
    tot, prod = 1.0, 1.0
    for k in range(1, N):
        pi = ((k - 1) * A[i][i] + (N - k) * A[i][j]) / (N - 1)
        pj = (k * A[j][i] + (N - k - 1) * A[j][j]) / (N - 1)
        prod *= math.exp(-beta * (pi - pj)); tot += prod
        if prod > 1e300: return 0.0
    return 1 / tot

def rules(b, chi=3.0):
    # extortioner for the donation game (R, S, T, P) = (b - 1, -1, b, 0), Press-Dyson form
    # with phi chosen so that p_CD = 1/2: s_X = chi * s_Y against any opponent
    phi = 0.5 / (1 + chi * b)
    ext = (1 - phi * (chi - 1) * (b - 1), 0.5, phi * (b + chi), 0)
    return {"ALLD": (0, 0, 0, 0), "ALLC": (1, 1, 1, 1), "TFT": (1, 0, 1, 0),
            "GTFT": (1, 1 / 3, 1, 1 / 3), "WSLS": (1, 0, 0, 1), "EXTORT": ext}

def abundance(names, b, N, beta=1.0, eps=0.01):
    D, R = (b - 1, -1, b, 0), rules(b)
    A = [[p06.payoffs(R[x], R[y], eps, *D)[0] for y in names] for x in names]
    n = len(names)
    T = [[fixation(A, i, j, N, beta) / (n - 1) if i != j else 0 for i in range(n)]
         for j in range(n)]                        # T[j][i]: from all-j to all-i
    for j in range(n): T[j][j] = 1 - sum(T[j])
    v = [1 / n] * n
    for _ in range(200000):
        w = [sum(v[j] * T[j][i] for j in range(n)) for i in range(n)]
        if max(abs(x - y) for x, y in zip(v, w)) < 1e-14: break
        v = w
    return v

NAMES = list(rules(3.0))
print("long-run share of time (N = 100, beta = 1, error 1 %)")
print("  b/c   " + "  ".join(f"{k:>6s}" for k in NAMES))
for b in (1.5, 2.0, 2.5, 3.0, 4.0, 6.0):
    v = abundance(NAMES, b, 100)
    print(f"  {b:3.1f}   " + "  ".join(f"{x:6.3f}" for x in v))
print("ALLD, ALLC, TFT, WSLS only (as in Imhof et al. 2007), b/c = 2.5 and 4:")
sub = ["ALLD", "ALLC", "TFT", "WSLS"]
for b in (2.5, 4.0):
    v = abundance(sub, b, 100)
    print(f"  b/c = {b}: " + ", ".join(f"{k} {x:.3f}" for k, x in zip(sub, v)))
print("population size and extortion, b/c = 3 (share of EXTORT; of ALLD)")
for N in (4, 10, 30, 100):
    v = abundance(NAMES, 3.0, N)
    print(f"  N = {N:3d}: EXTORT {v[5]:.3f}; ALLD {v[0]:.3f}; WSLS {v[4]:.3f}")
# long-run share of time (N = 100, beta = 1, error 1 %)
#   b/c     ALLD    ALLC     TFT    GTFT    WSLS  EXTORT
#   1.5    0.710   0.011   0.057   0.142   0.050   0.030
#   2.0    0.402   0.013   0.025   0.373   0.181   0.006
#   2.5    0.049   0.002   0.003   0.064   0.881   0.000
#   3.0    0.000   0.000   0.000   0.001   0.999   0.000
#   4.0    0.000   0.000   0.000   0.000   1.000   0.000
#   6.0    0.000   0.000   0.000   0.000   1.000   0.000
# ALLD, ALLC, TFT, WSLS only (as in Imhof et al. 2007), b/c = 2.5 and 4:
#   b/c = 2.5: ALLD 0.072, ALLC 0.004, TFT 0.007, WSLS 0.918
#   b/c = 4.0: ALLD 0.000, ALLC 0.000, TFT 0.000, WSLS 1.000
# population size and extortion, b/c = 3 (share of EXTORT; of ALLD)
#   N =   4: EXTORT 0.148; ALLD 0.307; WSLS 0.193
#   N =  10: EXTORT 0.036; ALLD 0.223; WSLS 0.380
#   N =  30: EXTORT 0.002; ALLD 0.069; WSLS 0.778
#   N = 100: EXTORT 0.000; ALLD 0.000; WSLS 0.999

在 100 人的种群、1% 的错误下,b/c=1.5b/c = 1.5 时种群 71% 的时间停在永远背叛;b/c=2b/c = 2 时永远背叛与慷慨一报还一报各占约四成(0.402、0.373),赢定输移占 0.181;b/c=2.5b/c = 2.5 时赢定输移占 0.881,b/c≥3b/c \ge 3 时几乎全部时间都是赢定输移。只比较永远背叛、永远合作、一报还一报与赢定输移四种策略时,结果与因霍夫、富登博格与诺瓦克 2007 年的结论一致:收益—代价比低于某个临界值时永远背叛被选择,高于临界值时赢定输移被选择,一报还一报本身从不占多数 (Imhof et al. 2007)。

勒索策略在 100 人的种群中几乎从不出现(b/c=3b/c = 3 时为 0.000);种群越小它越常见:4 人时占 14.8%,10 人时 3.6%。这与希尔贝、诺瓦克与西格蒙德 2013 年的结论一致:勒索在小种群中可以获得优势,在大种群中不能 (Hilbe et al. 2013)。斯图尔特与普洛特金同年发现,在演化中稳健的是零行列式策略中「慷慨」的一类:勒索者彼此相遇时表现很差 (Stewart and Plotkin 2013)。

左:b = 3、c = 1 的捐献博弈中,三种策略与自己副本的长期平均收益随错误率的变化。一报还一报在任何正的错误率下都只得 1(全合作为 2);慷慨一报还一报与赢定输移在错误率低于约 1% 时接近 2。右:100 人种群、1% 错误下,长期中种群停留在每种策略上的时间比例随 b/c 的变化。

证据

人类实验。 富登博格、兰德与德雷伯 2012 年在有执行错误的重复囚徒困境实验中发现,参与者的合作维持得相当好,最常见的成功策略「慢于发怒、快于原谅」:对单次背叛不立即报复,并很快回到合作 (Fudenberg et al. 2012)。达尔博与弗雷谢特 2018 年综述了大量无限重复博弈实验:继续互动的概率越高,合作越多;但合作作为均衡「可能」并不够,背叛策略的吸引域大小能更好地预测实际的合作水平 (Dal Bó and Fréchette 2018)。这与第一章右下图的含义一致。

假说 · 尚未验证

纠错宽容度假说(尚未确立)。作者推测,在交付偶尔出错(延迟、错发)的长期合作关系中,事先约定「第一次失误只沟通、不扣罚」的合作方,比约定「每次失误都按合同扣罚」的合作方,在一年后的合作存续率更高,前提是失误率低于约 5%;失误率很高时差别消失。可以检验的预测来自对多组合作关系的前瞻性观察或随机化的模拟实验。如果两种约定的存续率没有差别,或者严格扣罚的存续率更高,这一假说就被推翻。

本章小结

一报还一报在阿克塞尔罗德的两次锦标赛中胜出,但它从不在单局中击败对手,名次依赖参赛构成。有执行错误时,两个一报还一报陷入报复循环,长期收益降到四种结果的平均值。慷慨一报还一报的原谅概率有上限 min⁡{1−(T−R)/(R−S),(R−P)/(T−P)}\min\{1 – (T – R)/(R – S), (R – P)/(T – P)\},捐献博弈中为 1−c/b1 – c/b;赢定输移在 b/c>2b/c > 2 时稳健,在演化中占优。零行列式的勒索策略在每局中都不吃亏,但在大种群的演化中被淘汰。直接互惠需要同一对个体反复相遇;下一章讨论另一种情形:帮助一个以后不会再见的人。

参考文献

Axelrod, Robert. 1980a. “Effective Choice in the Prisoner’s Dilemma.” Journal of Conflict Resolution 24 (1): 3–25. https://doi.org/10.1177/002200278002400101.
Axelrod, Robert. 1980b. “More Effective Choice in the Prisoner’s Dilemma.” Journal of Conflict Resolution 24 (3): 379–403. https://doi.org/10.1177/002200278002400301.
Axelrod, Robert, and William D. Hamilton. 1981. “The Evolution of Cooperation.” Science 211 (4489): 1390–96. https://doi.org/10.1126/science.7466396.
Dal Bó, Pedro, and Guillaume R. Fréchette. 2018. “On the Determinants of Cooperation in Infinitely Repeated Games: A Survey.” Journal of Economic Literature 56 (1): 60–114. https://doi.org/10.1257/jel.20160980.
Fudenberg, Drew, David G. Rand, and Anna Dreber. 2012. “Slow to Anger and Fast to Forgive: Cooperation in an Uncertain World.” American Economic Review 102 (2): 720–49. https://doi.org/10.1257/aer.102.2.720.
Hilbe, Christian, Martin A. Nowak, and Karl Sigmund. 2013. “Evolution of Extortion in Iterated Prisoner’s Dilemma Games.” Proceedings of the National Academy of Sciences 110 (17): 6913–18. https://doi.org/10.1073/pnas.1214834110.
Imhof, Lorens A., Drew Fudenberg, and Martin A. Nowak. 2007. “Tit-for-Tat or Win-Stay, Lose-Shift?” Journal of Theoretical Biology 247 (3): 574–80. https://doi.org/10.1016/j.jtbi.2007.03.027.
Kraines, David, and Vivian Kraines. 1989. “Pavlov and the Prisoner’s Dilemma.” Theory and Decision 26 (1): 47–79. https://doi.org/10.1007/BF00134056.
Molander, Per. 1985. “The Optimal Level of Generosity in a Selfish, Uncertain Environment.” Journal of Conflict Resolution 29 (4): 611–18. https://doi.org/10.1177/0022002785029004004.
Nowak, Martin A., and Karl Sigmund. 1992. “Tit for Tat in Heterogeneous Populations.” Nature 355 (6357): 250–53. https://doi.org/10.1038/355250a0.
Nowak, Martin, and Karl Sigmund. 1993. “A Strategy of Win-Stay, Lose-Shift That Outperforms Tit-for-Tat in the Prisoner’s Dilemma Game.” Nature 364 (6432): 56–58. https://doi.org/10.1038/364056a0.
Press, William H., and Freeman J. Dyson. 2012. “Iterated Prisoner’s Dilemma Contains Strategies That Dominate Any Evolutionary Opponent.” Proceedings of the National Academy of Sciences 109 (26): 10409–13. https://doi.org/10.1073/pnas.1206569109.
Stewart, Alexander J., and Joshua B. Plotkin. 2013. “From Extortion to Generosity, Evolution in the Iterated Prisoner’s Dilemma.” Proceedings of the National Academy of Sciences 110 (38): 15348–53. https://doi.org/10.1073/pnas.1306246110.
Trivers, Robert L. 1971. “The Evolution of Reciprocal Altruism.” The Quarterly Review of Biology 46 (1): 35–57. https://doi.org/10.1086/406755.

Fan, Y. (2026). Evolutionary Game Theory, Direct and Indirect Reciprocity and the Emergence of Cooperation: From Hamilton's Rule and Axelrod's Tournaments to Reputation and Costly Punishment, Read Beside Classical Confucian and Mohist Ethics [演化博弈、直接与间接互惠及合作涌现的数学模型] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23788551 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).