5D Library › Evolutionary Game Theory, Direct and Indirect Reciprocity and the Emergence of Cooperation · 中文

Chapter 1 Replicator Dynamics: Why Cooperation Loses in One-Shot Anonymous Games

From individual choices to population frequencies

Classical game theory asks how rational players will choose. Evolutionary game theory asks a different question: if strategies are not “chosen” but carried by individuals and copied in proportion to their payoffs, which strategies will remain in the population after many generations? “Payoff” is here interpreted as fitness, the number of offspring left (or of imitators gained); the players need no rationality and need not know the structure of the game (Maynard Smith 1982; Hofbauer and Sigmund 1998).

Evidence

Evolutionarily stable strategies and the replicator equation. In 1973 Maynard Smith and Price introduced the concept of an evolutionarily stable strategy (ESS): a strategy is evolutionarily stable if, when the whole population adopts it, no small group of mutants using another strategy can invade. In their paper they used computer simulation to compare five fighting strategies of “limited war” (Maynard Smith and Price 1973); the two-strategy “hawk–dove game” is a later simplification that is discussed systematically in Maynard Smith’s book of 1982 (Maynard Smith 1982). In 1978 Taylor and Jonker gave the differential equation describing continuous change in strategy frequencies and showed that an ESS is an asymptotically stable rest point of these dynamics (Taylor and Jonker 1978); the name “replicator equation” comes from a paper by Schuster and Sigmund in 1983 (Schuster and Sigmund 1983). The converse does not hold: an asymptotically stable rest point need not be an ESS (Hofbauer and Sigmund 1998).

Calculation

Let xix_i be the share of strategy ii in the population, fi(x)=∑jaijxjf_i(x) = \sum_j a_{ij} x_j its average payoff against the population, and f‾(x)=∑ixifi(x)\bar f(x) = \sum_i x_i f_i(x) the population’s average payoff. The replicator equation is

ẋi=xi[fi(x)−f‾(x)]\dot x_i = x_i\,[f_i(x) – \bar f(x)]

Strategies whose payoff is above average grow; those below average shrink. With only two strategies, writing xx for the share of strategy 1, the equation becomes

ẋ=x(1−x)[f1(x)−f2(x)]\dot x = x(1 – x)\,[f_1(x) – f_2(x)]

The rest points are x=0x = 0, x=1x = 1 and, if it lies between 0 and 1, the interior point x*=(d−b)/(a−b−c+d)x^* = (d – b)/(a – b – c + d) at which the two strategies earn the same, where a,ba, b are the payoffs of strategy 1 against strategies 1 and 2, and c,dc, d those of strategy 2.

Three games, three phase portraits

The three simplest 2 × 2 games illustrate the three kinds of outcome the replicator equation can produce. The prisoner’s dilemma is written as a “donation game”: a cooperator pays a cost cc to give the other player a benefit b>cb > c; a defector pays nothing. We take b=3b = 3, c=1c = 1. In the hawk–dove game the resource is worth V=2V = 2 and the injury cost of a fight between two “hawks” is 4. In the stag hunt two hunters who hunt stag together get 4 each, a lone stag hunter gets 0, and hunting hare yields 3 whatever the others do. All payoffs are illustrative.

Calculation

# Replicator dynamics dx/dt = x (1 - x) [f_1(x) - f_2(x)] for three symmetric 2 x 2 games.
# Row payoffs: strategy 1 meeting 1 gets a, meeting 2 gets b; strategy 2 gets c and d.
#   Prisoner's dilemma (donation game, b = 3, c = 1):
#       C vs C 2, C vs D -1, D vs C 3, D vs D 0
#   Hawk-dove (resource V = 2, injury cost 4): H vs H -1, H vs D 2, D vs H 0, D vs D 1
#   Stag hunt: stag vs stag 4, stag vs hare 0, hare vs anything 3
# Payoffs are illustrative. Strategy 1 is listed first; x is its share.
GAMES = {"prisoner's dilemma (C, D)": (2, -1, 3, 0),
         "hawk-dove (H, D)": (-1, 2, 0, 1),
         "stag hunt (S, H)": (4, 0, 3, 3)}

def xdot(x, g):
    a, b, c, d = g
    return x * (1 - x) * ((a * x + b * (1 - x)) - (c * x + d * (1 - x)))

def rest_points(g):
    a, b, c, d = g
    pts = [0.0, 1.0]
    den = a - b - c + d
    if den != 0 and 0 < (d - b) / den < 1:
        pts.append((d - b) / den)
    return sorted(pts)

def stable(x, g, h=1e-6):           # sign of the slope of xdot at the rest point
    lo, hi = max(x - h, 0), min(x + h, 1)
    return (xdot(hi, g) - xdot(lo, g)) / (hi - lo) < 0

def ess(x, g):                       # Maynard Smith & Price condition, checked against
    a, b, c, d = g                   # every mutant mix q != x on a fine grid
    f = lambda p, q: p * q * a + p * (1 - q) * b + (1 - p) * q * c + (1 - p) * (1 - q) * d
    for k in range(1001):
        q = k / 1000
        if abs(q - x) < 1e-9: continue
        if f(x, x) < f(q, x) - 1e-12: return False
        if abs(f(x, x) - f(q, x)) < 1e-12 and f(x, q) <= f(q, q): return False
    return True

def run(x, g, t=40.0, dt=0.01):     # fourth-order Runge-Kutta
    for _ in range(int(t / dt)):
        k1 = xdot(x, g); k2 = xdot(x + dt / 2 * k1, g)
        k3 = xdot(x + dt / 2 * k2, g); k4 = xdot(x + dt * k3, g)
        x += dt / 6 * (k1 + 2 * k2 + 2 * k3 + k4)
    return x

for name, g in GAMES.items():
    print(name)
    for x in rest_points(g):
        print(f"   rest point x = {x:.3f}: {'stable' if stable(x, g) else 'unstable'}"
              f"{', ESS' if ess(x, g) else ''}")
    ends = [run(x0, g) for x0 in (0.05, 0.25, 0.5, 0.74, 0.76, 0.95)]
    print("   start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: "
          + " ".join(f"{e:.2f}" for e in ends))

# Three strategies in the repeated donation game (b = 3, c = 1) with m = 1/(1 - w)
# expected rounds: ALLD, TFT, ALLC. Payoffs per game: TFT/ALLC with each other or
# themselves m(b-c), TFT vs ALLD -c, ALLD vs TFT b, ALLD vs ALLC m b, ALLC vs ALLD -m c,
# ALLD vs ALLD 0.
def basin(m, n=40, t=150.0, dt=0.1):
    A = [[0, 3, 3 * m], [-1, 2 * m, 2 * m], [-m, 2 * m, 2 * m]]   # rows ALLD, TFT, ALLC
    def f(x):
        p = [sum(A[i][j] * x[j] for j in range(3)) for i in range(3)]
        av = sum(x[i] * p[i] for i in range(3))
        return [x[i] * (p[i] - av) for i in range(3)]
    won = tot = 0
    for i in range(1, n):
        for j in range(1, n - i):
            x = [i / n, j / n, 1 - (i + j) / n]
            for _ in range(int(t / dt)):
                d = f(x); x = [max(0.0, x[k] + dt * d[k]) for k in range(3)]
                s = sum(x); x = [v / s for v in x]
                if max(abs(v) for v in d) < 1e-6: break
            tot += 1; won += x[0] > 0.99
    return won / tot

print("repeated game, share of starting mixes that end at ALLD (grid of 741 starts):")
for m in (1.2, 2, 5, 10):
    print(f"   expected rounds m = {m:>4}: {basin(m):.2f}")
# prisoner's dilemma (C, D)
#    rest point x = 0.000: stable, ESS
#    rest point x = 1.000: unstable
#    start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: 0.00 0.00 0.00 0.00 0.00 0.00
# hawk-dove (H, D)
#    rest point x = 0.000: unstable
#    rest point x = 0.500: stable, ESS
#    rest point x = 1.000: unstable
#    start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: 0.50 0.50 0.50 0.50 0.50 0.50
# stag hunt (S, H)
#    rest point x = 0.000: stable, ESS
#    rest point x = 0.750: unstable
#    rest point x = 1.000: stable, ESS
#    start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: 0.00 0.00 0.00 0.00 1.00 1.00
# repeated game, share of starting mixes that end at ALLD (grid of 741 starts):
#    expected rounds m =  1.2: 1.00
#    expected rounds m =    2: 0.74
#    expected rounds m =    5: 0.20
#    expected rounds m =   10: 0.10

The prisoner’s dilemma has a single stable rest point, x=0x = 0: everybody defects. Whether we start from 5 % or 95 % cooperators, cooperators have vanished by t=40t = 40. The reason is written in the equation: in the donation game a cooperator earns cc less than a defector whoever the partner is, so fC−fD=−cf_C – f_D = -c is always negative. This is what “in one-shot anonymous interactions defectors eliminate cooperators” means: with no repetition, no reputation and no structure, cooperation earns no return.

But the conclusion holds only for the prisoner’s dilemma. In the hawk–dove game both end points are unstable and the population settles at x*=V/C=2/4=0.5x^* = V/C = 2/4 = 0.5 (with CC the injury cost): when hawks are many they injure each other; when they are few they take advantage. This mixed state is also an ESS. In the stag hunt both end points are ESSs and the interior point x*=0.75x^* = 0.75 is unstable: if fewer than 75 % hunt stag at the start, everyone ends up hunting hare; if more, everyone ends up hunting stag. Starting from 0.74 and from 0.76 leads to opposite outcomes. The stag hunt is often used to illustrate another difficulty of cooperation: cooperation itself is stable, and the hard part is getting from “nobody cooperates” to “everybody cooperates”.

Phase portraits of the replicator equation. Top left: the prisoner’s dilemma (donation game, b = 3, c = 1); the share of cooperators x always falls. Top right: the hawk–dove game; the population converges to the stable interior point 0.5. Bottom left: the stag hunt; 0.75 is an unstable dividing point. Filled circles are stable rest points, open circles unstable ones, and arrows show the direction in which x moves. Bottom right: in the repeated donation game, the share of 406 starting mixes (a grid with 30 steps per side) that end entirely at always defect (ALLD) falls as the expected number of rounds 1/(1 - w) rises; the dotted line marks w = c/b.

The last part of the program anticipates Chapter 3. If the same donation game is repeated, with probability ww of another round after each round, the expected number of rounds is m=1/(1−w)m = 1/(1 – w). In a population of always defect (ALLD), tit for tat (TFT) and always cooperate (ALLC), every starting state goes to ALLD when m=1.2m = 1.2 (w≈0.17w \approx 0.17, below c/b=1/3c/b = 1/3); when m=2m = 2, 74 % of starting states still go to ALLD; when m=5m = 5 and m=10m = 10 this share falls to 20 % and 10 %. Repeated interaction does not abolish the basin of attraction of defection; it shrinks it. A population made up mostly of defectors still stays with defection.

Note

Assumptions of the replicator equation. The population is infinite; individuals meet at random; payoffs translate directly into growth rates; strategies are inherited or imitated exactly, without mutation. Each assumption can be relaxed, and relaxing it may change the conclusions. When “replication” is interpreted as imitation learning in people, the equation describes “practices with higher payoffs being adopted by more people”; this is a model, and people do not always imitate those who earn most.

Finite populations: drift and fixation probabilities

Real populations are finite. In a finite population a cooperator mutant has some chance of taking over the whole population by chance (genetic drift), even if cooperators lose out at every step. The quantity that measures this is the fixation probability ρ\rho, the probability that a single mutant eventually replaces everyone. A neutral mutant has fixation probability 1/N1/N; ρ>1/N\rho > 1/N means that selection favours the mutant and ρ<1/N\rho < 1/N that selection opposes it (Nowak et al. 2004).

Calculation

# Finite populations: the Moran process. N individuals; each step one is chosen to
# reproduce with probability proportional to fitness 1 - s + s * (average payoff from
# games with all others), and its offspring replaces a randomly chosen individual. rho =
# probability that a single mutant takes over (neutral benchmark 1/N). Closed form (e.g.
# Nowak et al. 2004):
#     rho = 1 / (1 + sum_{k=1}^{N-1} prod_{j=1}^{k} g_j / f_j),
# f_j, g_j = fitness of mutant and resident when there are j mutants.
import random
rng = random.Random(2)

def fit(j, N, g, s):
    a, b, c, d = g                      # mutant row first: a, b; resident row: c, d
    pm = (a * (j - 1) + b * (N - j)) / (N - 1)
    pr = (c * j + d * (N - j - 1)) / (N - 1)
    return 1 - s + s * pm, 1 - s + s * pr

def rho(N, g, s):
    tot, prod = 1.0, 1.0
    for k in range(1, N):
        fm, fr = fit(k, N, g, s); prod *= fr / fm; tot += prod
    return 1 / tot

def simulate(N, g, s, runs):
    won = 0
    for _ in range(runs):
        j = 1
        while 0 < j < N:
            fm, fr = fit(j, N, g, s)
            birth_m = rng.random() < j * fm / (j * fm + (N - j) * fr)
            death_m = rng.random() < j / N
            j += birth_m - death_m
        won += j == N
    return won / runs

PD = (2, -1, 3, 0)       # one-shot anonymous donation game, b = 3, c = 1 (C row first)
DC = (0, 3, -1, 2)       # the same game with D as the mutant
print("one-shot donation game, b = 3, c = 1; rho x N (1 = neutral)")
print("   N     s     cooperator mutant   defector mutant")
for N in (10, 50, 200):
    for s in (0.01, 0.1):
        rc, rd = rho(N, PD, s) * N, rho(N, DC, s) * N
        print(f"{N:4d}  {s:4.2f}        {rc:8.4f}        {rd:9.4f}")
print(f"simulation check, N = 20, s = 0.1, 40,000 runs:\n   cooperator rho x N ="
      f" {simulate(20, PD, 0.1, 40000) * 20:.3f} (formula {rho(20, PD, 0.1) * 20:.3f})")
# Stag hunt, payoffs a, b, c, d = 4, 0, 3, 3 (stag row first): the unstable point is x*
# = (d - b)/(a - b - c + d) = 0.75. The one-third law (Nowak et al. 2004): under weak
# selection in a large population, a single stag hunter is favoured (rho > 1/N) iff x* <
# 1/3.
print("stag hunt, weak selection s = 0.005, N = 100: rho x N for a single stag hunter")
for a in (4, 6, 9, 10, 12, 20):
    g = (a, 0, 3, 3); xs = 3 / a
    rn = 100 * rho(100, g, 0.005)
    print(f"   stag payoff {a:>2}: x* = {xs:.3f}, rho x N = {rn:.3f}")
# one-shot donation game, b = 3, c = 1; rho x N (1 = neutral)
#    N     s     cooperator mutant   defector mutant
#   10  0.01          0.9408           1.0608
#   10  0.10          0.4941           1.6585
#   50  0.01          0.7611           1.2802
#   50  0.10          0.0246           4.6693
#  200  0.01          0.3070           2.3145
#  200  0.10          0.0000          17.1476
# simulation check, N = 20, s = 0.1, 40,000 runs:
#    cooperator rho x N = 0.257 (formula 0.253)
# stag hunt, weak selection s = 0.005, N = 100: rho x N for a single stag hunter
#    stag payoff  4: x* = 0.750, rho x N = 0.653
#    stag payoff  6: x* = 0.500, rho x N = 0.773
#    stag payoff  9: x* = 0.333, rho x N = 0.959
#    stag payoff 10: x* = 0.300, rho x N = 1.022
#    stag payoff 12: x* = 0.250, rho x N = 1.147
#    stag payoff 20: x* = 0.150, rho x N = 1.620

The program computes fixation probabilities for the Moran process (at each step one individual is chosen to reproduce in proportion to fitness and its offspring replaces a randomly chosen individual) and reports ρN\rho N, the multiple of the neutral value. In the one-shot anonymous donation game a cooperator’s ρN\rho N is always below 1 and a defector’s always above 1. The larger the population and the stronger the selection, the larger the gap: with N=10N = 10 and selection strength s=0.01s = 0.01 a cooperator’s fixation probability is only 6 % below neutral (0.94); with N=200N = 200 and s=0.1s = 0.1 it is effectively zero, while a defector’s is 17 times the neutral value. A simulation with N=20N = 20 and s=0.1s = 0.1 (40,000 runs) gives 0.257, in agreement with the formula’s 0.253. In small populations a cooperator occasionally becomes fixed by luck, but the direction of selection is always against it.

Evidence

The one-third law. Nowak, Sasaki, Taylor and Fudenberg showed in 2004 that in large finite populations under weak selection, for a bistable game such as the stag hunt, a single mutant of the cooperative strategy is favoured by selection (ρ>1/N\rho > 1/N) if and only if the unstable point x*x^* is below 1/3. So even when cooperation is stable in the replicator equation, whether it can spread from scratch in a finite population also depends on whether its basin of attraction is large enough (Nowak et al. 2004).

The last part of the program checks this law. With a stag payoff of 4, x*=0.75x^* = 0.75 and ρN=0.65\rho N = 0.65; with a stag payoff of 12, x*=0.25x^* = 0.25 and ρN=1.15\rho N = 1.15. The dividing line lies close to x*=1/3x^* = 1/3: at x*=0.333x^* = 0.333, ρN=0.96\rho N = 0.96, and at x*=0.30x^* = 0.30 it is 1.02. That ρN\rho N falls slightly below 1 exactly at 1/3 when N=100N = 100 is a finite-population correction, which shrinks as the population grows. Chapter 2 shows that the one-third law is one of the three criteria Nowak used to compare the five mechanisms.

Hypothesis · not established

Basin-of-attraction hypothesis (not yet established). The author conjectures that whether a newly founded cooperative group (for instance a newly formed volunteer community) develops a stable norm of mutual help depends mainly on whether the share of founding members “willing to give first” exceeds some threshold, rather than on the average willingness to cooperate; the position of the threshold is set by the payoff structure of the collaboration, much as x*x^* is in the stag hunt. A testable prediction is that across many newly formed online collaboration groups with identical rules, the relation between the share of active contributors in the first two weeks and the level of contribution after six months is S-shaped (threshold-like) rather than linear. If the relation is linear, or the early share has no predictive power, the hypothesis is refuted.

Chapter summary

The replicator equation ẋi=xi[fi(x)−f‾(x)]\dot x_i = x_i[f_i(x) – \bar f(x)] makes strategies with above-average payoffs grow. In the one-shot anonymous prisoner’s dilemma defection is better against every partner, and cooperators must disappear; but in the hawk–dove game the two strategies coexist, and in the stag hunt the outcome depends on the starting point. An ESS is a stable point of the replicator dynamics, but not conversely. In finite populations a cooperator can occasionally become fixed by drift, but its fixation probability is below neutral; for bistable games the one-third law gives the condition for spreading from scratch. The next chapter asks what mechanisms can change the payoff structure of the prisoner’s dilemma itself.

References

Hofbauer, Josef, and Karl Sigmund. 1998. Evolutionary Games and Population Dynamics. Cambridge University Press.
Maynard Smith, John. 1982. Evolution and the Theory of Games. Cambridge University Press.
Maynard Smith, John, and George R. Price. 1973. “The Logic of Animal Conflict.” Nature 246 (5427): 15–18. https://doi.org/10.1038/246015a0.
Nowak, Martin A., Akira Sasaki, Christine Taylor, and Drew Fudenberg. 2004. “Emergence of Cooperation and Evolutionary Stability in Finite Populations.” Nature 428 (6983): 646–50. https://doi.org/10.1038/nature02414.
Schuster, Peter, and Karl Sigmund. 1983. “Replicator Dynamics.” Journal of Theoretical Biology 100 (3): 533–38. https://doi.org/10.1016/0022-5193(83)90445-9.
Taylor, Peter D., and Leo B. Jonker. 1978. “Evolutionarily Stable Strategies and Game Dynamics.” Mathematical Biosciences 40 (1–2): 145–56. https://doi.org/10.1016/0025-5564(78)90077-9.

Fan, Y. (2026). Evolutionary Game Theory, Direct and Indirect Reciprocity and the Emergence of Cooperation: From Hamilton's Rule and Axelrod's Tournaments to Reputation and Costly Punishment, Read Beside Classical Confucian and Mohist Ethics [演化博弈、直接与间接互惠及合作涌现的数学模型] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23788551 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).