5D Library › 演化博弈、直接与间接互惠及合作涌现的数学模型 · English

第一章 复制子动力学:单次匿名博弈中合作为什么输

从个体的选择到种群的频率

经典博弈论问的是:理性的参与者会怎样选择?演化博弈论换了一个问法:如果策略不是被「选择」的,而是由个体携带、按收益多少被复制的,那么经过许多代之后,种群中哪些策略会留下来?这里的「收益」被解释为适合度,即留下后代(或被他人模仿)的多少;参与者不需要任何理性,也不需要知道博弈的结构 (Maynard Smith 1982; Hofbauer and Sigmund 1998)。

证据

演化稳定策略与复制子方程。 梅纳德·史密斯与普赖斯 1973 年提出演化稳定策略(ESS)的概念:如果整个种群都采用某一策略,任何少量的突变策略都无法侵入,这一策略就是演化稳定的。他们在论文中用计算机模拟比较了五种「有限战争」的打斗策略 (Maynard Smith and Price 1973);两策略的「鹰鸽博弈」是后来常用的简化形式,在梅纳德·史密斯 1982 年的专著中得到系统讨论 (Maynard Smith 1982)。泰勒与容克 1978 年给出了描述策略频率连续变化的微分方程,并证明 ESS 是这一动力学的渐近稳定的静止点 (Taylor and Jonker 1978);「复制子方程」这个名称来自舒斯特与西格蒙德 1983 年的文章 (Schuster and Sigmund 1983)。反过来不成立:渐近稳定的静止点不一定是 ESS (Hofbauer and Sigmund 1998)。

推导

设种群中策略 ii 的比例为 xix_i,它与种群相遇的平均收益为 fi(x)=∑jaijxjf_i(x) = \sum_j a_{ij} x_j,种群平均收益为 f‾(x)=∑ixifi(x)\bar f(x) = \sum_i x_i f_i(x)。复制子方程是

ẋi=xi[fi(x)−f‾(x)]\dot x_i = x_i\,[f_i(x) – \bar f(x)]

收益高于平均的策略比例增长,低于平均的比例下降。只有两种策略时,记策略 1 的比例为 xx,方程化为

ẋ=x(1−x)[f1(x)−f2(x)]\dot x = x(1 – x)\,[f_1(x) – f_2(x)]

静止点是 x=0x = 0、x=1x = 1,以及使两种策略收益相等的内点 x*=(d−b)/(a−b−c+d)x^* = (d – b)/(a – b – c + d)(若它在 0 与 1 之间),其中 a,ba, b 是策略 1 遇到策略 1、2 的收益,c,dc, d 是策略 2 遇到策略 1、2 的收益。

三种博弈,三种相图

用三个最简单的 2 × 2 博弈说明复制子方程能产生的三种结局。囚徒困境写成「捐献博弈」:合作者付出代价 cc,使对方得到 b>cb > c;背叛者什么也不付出。取 b=3b = 3、c=1c = 1。鹰鸽博弈中,资源价值 V=2V = 2,两只「鹰」相斗时受伤的代价为 4。猎鹿博弈中,两人一起猎鹿各得 4,单独猎鹿得 0,猎兔无论别人怎样都得 3。这些收益都是示意值。

推导

# Replicator dynamics dx/dt = x (1 - x) [f_1(x) - f_2(x)] for three symmetric 2 x 2 games.
# Row payoffs: strategy 1 meeting 1 gets a, meeting 2 gets b; strategy 2 gets c and d.
#   Prisoner's dilemma (donation game, b = 3, c = 1):
#       C vs C 2, C vs D -1, D vs C 3, D vs D 0
#   Hawk-dove (resource V = 2, injury cost 4): H vs H -1, H vs D 2, D vs H 0, D vs D 1
#   Stag hunt: stag vs stag 4, stag vs hare 0, hare vs anything 3
# Payoffs are illustrative. Strategy 1 is listed first; x is its share.
GAMES = {"prisoner's dilemma (C, D)": (2, -1, 3, 0),
         "hawk-dove (H, D)": (-1, 2, 0, 1),
         "stag hunt (S, H)": (4, 0, 3, 3)}

def xdot(x, g):
    a, b, c, d = g
    return x * (1 - x) * ((a * x + b * (1 - x)) - (c * x + d * (1 - x)))

def rest_points(g):
    a, b, c, d = g
    pts = [0.0, 1.0]
    den = a - b - c + d
    if den != 0 and 0 < (d - b) / den < 1:
        pts.append((d - b) / den)
    return sorted(pts)

def stable(x, g, h=1e-6):           # sign of the slope of xdot at the rest point
    lo, hi = max(x - h, 0), min(x + h, 1)
    return (xdot(hi, g) - xdot(lo, g)) / (hi - lo) < 0

def ess(x, g):                       # Maynard Smith & Price condition, checked against
    a, b, c, d = g                   # every mutant mix q != x on a fine grid
    f = lambda p, q: p * q * a + p * (1 - q) * b + (1 - p) * q * c + (1 - p) * (1 - q) * d
    for k in range(1001):
        q = k / 1000
        if abs(q - x) < 1e-9: continue
        if f(x, x) < f(q, x) - 1e-12: return False
        if abs(f(x, x) - f(q, x)) < 1e-12 and f(x, q) <= f(q, q): return False
    return True

def run(x, g, t=40.0, dt=0.01):     # fourth-order Runge-Kutta
    for _ in range(int(t / dt)):
        k1 = xdot(x, g); k2 = xdot(x + dt / 2 * k1, g)
        k3 = xdot(x + dt / 2 * k2, g); k4 = xdot(x + dt * k3, g)
        x += dt / 6 * (k1 + 2 * k2 + 2 * k3 + k4)
    return x

for name, g in GAMES.items():
    print(name)
    for x in rest_points(g):
        print(f"   rest point x = {x:.3f}: {'stable' if stable(x, g) else 'unstable'}"
              f"{', ESS' if ess(x, g) else ''}")
    ends = [run(x0, g) for x0 in (0.05, 0.25, 0.5, 0.74, 0.76, 0.95)]
    print("   start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: "
          + " ".join(f"{e:.2f}" for e in ends))

# Three strategies in the repeated donation game (b = 3, c = 1) with m = 1/(1 - w)
# expected rounds: ALLD, TFT, ALLC. Payoffs per game: TFT/ALLC with each other or
# themselves m(b-c), TFT vs ALLD -c, ALLD vs TFT b, ALLD vs ALLC m b, ALLC vs ALLD -m c,
# ALLD vs ALLD 0.
def basin(m, n=40, t=150.0, dt=0.1):
    A = [[0, 3, 3 * m], [-1, 2 * m, 2 * m], [-m, 2 * m, 2 * m]]   # rows ALLD, TFT, ALLC
    def f(x):
        p = [sum(A[i][j] * x[j] for j in range(3)) for i in range(3)]
        av = sum(x[i] * p[i] for i in range(3))
        return [x[i] * (p[i] - av) for i in range(3)]
    won = tot = 0
    for i in range(1, n):
        for j in range(1, n - i):
            x = [i / n, j / n, 1 - (i + j) / n]
            for _ in range(int(t / dt)):
                d = f(x); x = [max(0.0, x[k] + dt * d[k]) for k in range(3)]
                s = sum(x); x = [v / s for v in x]
                if max(abs(v) for v in d) < 1e-6: break
            tot += 1; won += x[0] > 0.99
    return won / tot

print("repeated game, share of starting mixes that end at ALLD (grid of 741 starts):")
for m in (1.2, 2, 5, 10):
    print(f"   expected rounds m = {m:>4}: {basin(m):.2f}")
# prisoner's dilemma (C, D)
#    rest point x = 0.000: stable, ESS
#    rest point x = 1.000: unstable
#    start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: 0.00 0.00 0.00 0.00 0.00 0.00
# hawk-dove (H, D)
#    rest point x = 0.000: unstable
#    rest point x = 0.500: stable, ESS
#    rest point x = 1.000: unstable
#    start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: 0.50 0.50 0.50 0.50 0.50 0.50
# stag hunt (S, H)
#    rest point x = 0.000: stable, ESS
#    rest point x = 0.750: unstable
#    rest point x = 1.000: stable, ESS
#    start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: 0.00 0.00 0.00 0.00 1.00 1.00
# repeated game, share of starting mixes that end at ALLD (grid of 741 starts):
#    expected rounds m =  1.2: 1.00
#    expected rounds m =    2: 0.74
#    expected rounds m =    5: 0.20
#    expected rounds m =   10: 0.10

囚徒困境中只有一个稳定的静止点:x=0x = 0,全体背叛。无论从 5% 还是 95% 的合作者出发,到 t=40t = 40 时合作者都已消失。原因写在方程里:在捐献博弈中,不论对方是谁,合作者的收益都比背叛者少 cc,所以 fC−fD=−cf_C – f_D = -c 恒为负。这就是「在单次匿名互动中背叛者淘汰合作者」的含义:没有重复、没有声誉、没有结构,合作没有任何回报。

但这个结论只对囚徒困境成立。在鹰鸽博弈中,两个端点都不稳定,种群停在 x*=V/C=2/4=0.5x^* = V/C = 2/4 = 0.5(CC 为受伤代价):鹰多了,鹰之间互相伤害;鹰少了,鹰占便宜。这个混合状态也是 ESS。在猎鹿博弈中,两个端点都是 ESS,内点 x*=0.75x^* = 0.75 不稳定:初始猎鹿者少于 75% 时,所有人最终都去猎兔;多于 75% 时,所有人都去猎鹿。从 0.74 与 0.76 出发,结局完全相反。猎鹿博弈常被用来说明合作的另一个难处:合作本身是稳定的,难的是从「大家都不合作」走到「大家都合作」。

复制子方程的相图。左上:囚徒困境(捐献博弈,b = 3、c = 1),合作者比例 x 永远下降。右上:鹰鸽博弈,种群收敛到内部的稳定点 0.5。左下:猎鹿博弈,0.75 是不稳定的分界点。实心圆为稳定静止点,空心圆为不稳定静止点,箭头为 x 变化的方向。右下:在重复的捐献博弈中,从 406 个初始比例(每条边 30 格的网格)出发、最后全部落入永远背叛(ALLD)的比例,随期望轮数 1/(1 - w) 增加而下降;虚线为 w = c/b。

程序的最后一部分预告了第三章:同样的捐献博弈若重复进行,每轮之后还有下一轮的概率为 ww,期望轮数为 m=1/(1−w)m = 1/(1 – w)。在永远背叛(ALLD)、一报还一报(TFT)与永远合作(ALLC)三种策略的种群中,m=1.2m = 1.2(w≈0.17w \approx 0.17,低于 c/b=1/3c/b = 1/3)时所有初始状态都走向 ALLD;m=2m = 2 时仍有 74% 的初始状态走向 ALLD;m=5m = 5 与 m=10m = 10 时这一比例降到 20% 与 10%。重复互动没有消灭背叛的吸引域,只是把它缩小了:一个以背叛者为主的种群仍然会停在背叛。

注

复制子方程的假设。 种群无限大;个体随机相遇;收益直接转化为增长率;策略原样遗传或被模仿,没有突变。每一条都可以放宽,放宽后结论可能改变。把「复制」解释为人类的模仿学习时,方程描述的是「收益高的做法被更多人采用」,这是一种模型,人并不总是模仿收益高的人。

有限种群:漂变与固定概率

真实的种群是有限的。在有限种群中,即使合作者在每一步都吃亏,一个合作者突变也有一定的机会靠偶然性(遗传漂变)占领整个种群。衡量这一点的量是固定概率 ρ\rho:一个突变个体最终取代全体的概率。中性突变的固定概率是 1/N1/N;ρ>1/N\rho > 1/N 表示选择偏爱这个突变,ρ<1/N\rho < 1/N 表示选择不利于它 (Nowak et al. 2004)。

推导

# Finite populations: the Moran process. N individuals; each step one is chosen to
# reproduce with probability proportional to fitness 1 - s + s * (average payoff from
# games with all others), and its offspring replaces a randomly chosen individual. rho =
# probability that a single mutant takes over (neutral benchmark 1/N). Closed form (e.g.
# Nowak et al. 2004):
#     rho = 1 / (1 + sum_{k=1}^{N-1} prod_{j=1}^{k} g_j / f_j),
# f_j, g_j = fitness of mutant and resident when there are j mutants.
import random
rng = random.Random(2)

def fit(j, N, g, s):
    a, b, c, d = g                      # mutant row first: a, b; resident row: c, d
    pm = (a * (j - 1) + b * (N - j)) / (N - 1)
    pr = (c * j + d * (N - j - 1)) / (N - 1)
    return 1 - s + s * pm, 1 - s + s * pr

def rho(N, g, s):
    tot, prod = 1.0, 1.0
    for k in range(1, N):
        fm, fr = fit(k, N, g, s); prod *= fr / fm; tot += prod
    return 1 / tot

def simulate(N, g, s, runs):
    won = 0
    for _ in range(runs):
        j = 1
        while 0 < j < N:
            fm, fr = fit(j, N, g, s)
            birth_m = rng.random() < j * fm / (j * fm + (N - j) * fr)
            death_m = rng.random() < j / N
            j += birth_m - death_m
        won += j == N
    return won / runs

PD = (2, -1, 3, 0)       # one-shot anonymous donation game, b = 3, c = 1 (C row first)
DC = (0, 3, -1, 2)       # the same game with D as the mutant
print("one-shot donation game, b = 3, c = 1; rho x N (1 = neutral)")
print("   N     s     cooperator mutant   defector mutant")
for N in (10, 50, 200):
    for s in (0.01, 0.1):
        rc, rd = rho(N, PD, s) * N, rho(N, DC, s) * N
        print(f"{N:4d}  {s:4.2f}        {rc:8.4f}        {rd:9.4f}")
print(f"simulation check, N = 20, s = 0.1, 40,000 runs:\n   cooperator rho x N ="
      f" {simulate(20, PD, 0.1, 40000) * 20:.3f} (formula {rho(20, PD, 0.1) * 20:.3f})")
# Stag hunt, payoffs a, b, c, d = 4, 0, 3, 3 (stag row first): the unstable point is x*
# = (d - b)/(a - b - c + d) = 0.75. The one-third law (Nowak et al. 2004): under weak
# selection in a large population, a single stag hunter is favoured (rho > 1/N) iff x* <
# 1/3.
print("stag hunt, weak selection s = 0.005, N = 100: rho x N for a single stag hunter")
for a in (4, 6, 9, 10, 12, 20):
    g = (a, 0, 3, 3); xs = 3 / a
    rn = 100 * rho(100, g, 0.005)
    print(f"   stag payoff {a:>2}: x* = {xs:.3f}, rho x N = {rn:.3f}")
# one-shot donation game, b = 3, c = 1; rho x N (1 = neutral)
#    N     s     cooperator mutant   defector mutant
#   10  0.01          0.9408           1.0608
#   10  0.10          0.4941           1.6585
#   50  0.01          0.7611           1.2802
#   50  0.10          0.0246           4.6693
#  200  0.01          0.3070           2.3145
#  200  0.10          0.0000          17.1476
# simulation check, N = 20, s = 0.1, 40,000 runs:
#    cooperator rho x N = 0.257 (formula 0.253)
# stag hunt, weak selection s = 0.005, N = 100: rho x N for a single stag hunter
#    stag payoff  4: x* = 0.750, rho x N = 0.653
#    stag payoff  6: x* = 0.500, rho x N = 0.773
#    stag payoff  9: x* = 0.333, rho x N = 0.959
#    stag payoff 10: x* = 0.300, rho x N = 1.022
#    stag payoff 12: x* = 0.250, rho x N = 1.147
#    stag payoff 20: x* = 0.150, rho x N = 1.620

程序用莫兰过程(每一步按适合度挑一个个体繁殖,后代随机替换一个个体)计算固定概率,并以 ρN\rho N 表示相对于中性的倍数。在单次匿名的捐献博弈中,合作者的 ρN\rho N 总是小于 1,背叛者的总是大于 1。种群越大、选择越强,差距越大:N=10N = 10、选择强度 s=0.01s = 0.01 时,合作者的固定概率只比中性低 6%(0.94);N=200N = 200、s=0.1s = 0.1 时,它实际上为零,而背叛者是中性的 17 倍。对 N=20N = 20、s=0.1s = 0.1 的模拟(40,000 次)得到 0.257,与公式的 0.253 相符。小种群中合作者偶尔能靠运气固定下来,但选择的方向始终不利于它。

证据

三分之一法则。 诺瓦克、佐佐木、泰勒与富登博格 2004 年证明:在大的有限种群中、弱选择下,对于猎鹿博弈这样的双稳态博弈,一个合作策略突变被选择偏爱(ρ>1/N\rho > 1/N),当且仅当不稳定点 x*x^* 小于 1/3。这意味着即使合作在复制子方程中是稳定的,在有限种群中能否从零开始扩散,还取决于它的吸引域是否足够大 (Nowak et al. 2004)。

程序的最后一部分检验这一法则:猎鹿收益为 4 时 x*=0.75x^* = 0.75,ρN=0.65\rho N = 0.65;猎鹿收益升到 12 时 x*=0.25x^* = 0.25,ρN=1.15\rho N = 1.15。分界点在 x*=1/3x^* = 1/3 附近:x*=0.333x^* = 0.333 时 ρN=0.96\rho N = 0.96,x*=0.30x^* = 0.30 时为 1.02。N=100N = 100 时恰在 1/3 处略低于 1,是有限种群的修正,种群越大越接近 1/3。第二章会看到,「三分之一法则」正是诺瓦克比较五种机制时使用的三个标准之一。

假说 · 尚未验证

吸引域假说(尚未确立)。作者推测,一个新成立的协作团体(例如新组建的志愿者社群)能否形成稳定的互助规范,主要取决于最初一批成员中「愿意先付出」的人是否超过某个门槛,而不是取决于平均的合作意愿;门槛的位置由协作的收益结构决定,与猎鹿博弈的 x*x^* 相似。可以检验的预测是:在多个新组建、规则相同的线上协作小组中,前两周主动贡献者的比例与六个月后的贡献水平之间呈 S 形(阈值型)关系,而不是线性关系。如果关系是线性的,或者早期比例没有预测力,这一假说就被推翻。

本章小结

复制子方程 ẋi=xi[fi(x)−f‾(x)]\dot x_i = x_i[f_i(x) – \bar f(x)] 让收益高于平均的策略增长。在单次匿名的囚徒困境中,背叛对每个对手都更好,合作者必然消失;但鹰鸽博弈中两种策略共存,猎鹿博弈中结局取决于起点。ESS 是复制子动力学的稳定点,反之不然。在有限种群中,合作者偶尔可以靠漂变固定下来,但固定概率低于中性;对双稳态博弈,三分之一法则给出了从零开始扩散的条件。下一章问:什么样的机制能改变囚徒困境本身的收益结构?

参考文献

Hofbauer, Josef, and Karl Sigmund. 1998. Evolutionary Games and Population Dynamics. Cambridge University Press.
Maynard Smith, John. 1982. Evolution and the Theory of Games. Cambridge University Press.
Maynard Smith, John, and George R. Price. 1973. “The Logic of Animal Conflict.” Nature 246 (5427): 15–18. https://doi.org/10.1038/246015a0.
Nowak, Martin A., Akira Sasaki, Christine Taylor, and Drew Fudenberg. 2004. “Emergence of Cooperation and Evolutionary Stability in Finite Populations.” Nature 428 (6983): 646–50. https://doi.org/10.1038/nature02414.
Schuster, Peter, and Karl Sigmund. 1983. “Replicator Dynamics.” Journal of Theoretical Biology 100 (3): 533–38. https://doi.org/10.1016/0022-5193(83)90445-9.
Taylor, Peter D., and Leo B. Jonker. 1978. “Evolutionarily Stable Strategies and Game Dynamics.” Mathematical Biosciences 40 (1–2): 145–56. https://doi.org/10.1016/0025-5564(78)90077-9.

Fan, Y. (2026). Evolutionary Game Theory, Direct and Indirect Reciprocity and the Emergence of Cooperation: From Hamilton's Rule and Axelrod's Tournaments to Reputation and Costly Punishment, Read Beside Classical Confucian and Mohist Ethics [演化博弈、直接与间接互惠及合作涌现的数学模型] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23788551 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).