5D Library › Evolutionary Game Theory, Direct and Indirect Reciprocity and the Emergence of Cooperation · 中文
Chapter 1 Replicator Dynamics: Why Cooperation Loses in One-Shot Anonymous Games
From individual choices to population frequencies
Classical game theory asks how rational players will choose. Evolutionary game theory asks a different question: if strategies are not “chosen” but carried by individuals and copied in proportion to their payoffs, which strategies will remain in the population after many generations? “Payoff” is here interpreted as fitness, the number of offspring left (or of imitators gained); the players need no rationality and need not know the structure of the game (Maynard Smith 1982; Hofbauer and Sigmund 1998).
Evidence
Evolutionarily stable strategies and the replicator equation. In 1973 Maynard Smith and Price introduced the concept of an evolutionarily stable strategy (ESS): a strategy is evolutionarily stable if, when the whole population adopts it, no small group of mutants using another strategy can invade. In their paper they used computer simulation to compare five fighting strategies of “limited war” (Maynard Smith and Price 1973); the two-strategy “hawk–dove game” is a later simplification that is discussed systematically in Maynard Smith’s book of 1982 (Maynard Smith 1982). In 1978 Taylor and Jonker gave the differential equation describing continuous change in strategy frequencies and showed that an ESS is an asymptotically stable rest point of these dynamics (Taylor and Jonker 1978); the name “replicator equation” comes from a paper by Schuster and Sigmund in 1983 (Schuster and Sigmund 1983). The converse does not hold: an asymptotically stable rest point need not be an ESS (Hofbauer and Sigmund 1998).
Calculation
Let be the share of strategy in the population, its average payoff against the population, and the population’s average payoff. The replicator equation is
Strategies whose payoff is above average grow; those below average shrink. With only two strategies, writing for the share of strategy 1, the equation becomes
The rest points are , and, if it lies between 0 and 1, the interior point at which the two strategies earn the same, where are the payoffs of strategy 1 against strategies 1 and 2, and those of strategy 2.
Three games, three phase portraits
The three simplest 2 × 2 games illustrate the three kinds of outcome the replicator equation can produce. The prisoner’s dilemma is written as a “donation game”: a cooperator pays a cost to give the other player a benefit ; a defector pays nothing. We take , . In the hawk–dove game the resource is worth and the injury cost of a fight between two “hawks” is 4. In the stag hunt two hunters who hunt stag together get 4 each, a lone stag hunter gets 0, and hunting hare yields 3 whatever the others do. All payoffs are illustrative.
Calculation
# Replicator dynamics dx/dt = x (1 - x) [f_1(x) - f_2(x)] for three symmetric 2 x 2 games.
# Row payoffs: strategy 1 meeting 1 gets a, meeting 2 gets b; strategy 2 gets c and d.
# Prisoner's dilemma (donation game, b = 3, c = 1):
# C vs C 2, C vs D -1, D vs C 3, D vs D 0
# Hawk-dove (resource V = 2, injury cost 4): H vs H -1, H vs D 2, D vs H 0, D vs D 1
# Stag hunt: stag vs stag 4, stag vs hare 0, hare vs anything 3
# Payoffs are illustrative. Strategy 1 is listed first; x is its share.
GAMES = {"prisoner's dilemma (C, D)": (2, -1, 3, 0),
"hawk-dove (H, D)": (-1, 2, 0, 1),
"stag hunt (S, H)": (4, 0, 3, 3)}
def xdot(x, g):
a, b, c, d = g
return x * (1 - x) * ((a * x + b * (1 - x)) - (c * x + d * (1 - x)))
def rest_points(g):
a, b, c, d = g
pts = [0.0, 1.0]
den = a - b - c + d
if den != 0 and 0 < (d - b) / den < 1:
pts.append((d - b) / den)
return sorted(pts)
def stable(x, g, h=1e-6): # sign of the slope of xdot at the rest point
lo, hi = max(x - h, 0), min(x + h, 1)
return (xdot(hi, g) - xdot(lo, g)) / (hi - lo) < 0
def ess(x, g): # Maynard Smith & Price condition, checked against
a, b, c, d = g # every mutant mix q != x on a fine grid
f = lambda p, q: p * q * a + p * (1 - q) * b + (1 - p) * q * c + (1 - p) * (1 - q) * d
for k in range(1001):
q = k / 1000
if abs(q - x) < 1e-9: continue
if f(x, x) < f(q, x) - 1e-12: return False
if abs(f(x, x) - f(q, x)) < 1e-12 and f(x, q) <= f(q, q): return False
return True
def run(x, g, t=40.0, dt=0.01): # fourth-order Runge-Kutta
for _ in range(int(t / dt)):
k1 = xdot(x, g); k2 = xdot(x + dt / 2 * k1, g)
k3 = xdot(x + dt / 2 * k2, g); k4 = xdot(x + dt * k3, g)
x += dt / 6 * (k1 + 2 * k2 + 2 * k3 + k4)
return x
for name, g in GAMES.items():
print(name)
for x in rest_points(g):
print(f" rest point x = {x:.3f}: {'stable' if stable(x, g) else 'unstable'}"
f"{', ESS' if ess(x, g) else ''}")
ends = [run(x0, g) for x0 in (0.05, 0.25, 0.5, 0.74, 0.76, 0.95)]
print(" start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: "
+ " ".join(f"{e:.2f}" for e in ends))
# Three strategies in the repeated donation game (b = 3, c = 1) with m = 1/(1 - w)
# expected rounds: ALLD, TFT, ALLC. Payoffs per game: TFT/ALLC with each other or
# themselves m(b-c), TFT vs ALLD -c, ALLD vs TFT b, ALLD vs ALLC m b, ALLC vs ALLD -m c,
# ALLD vs ALLD 0.
def basin(m, n=40, t=150.0, dt=0.1):
A = [[0, 3, 3 * m], [-1, 2 * m, 2 * m], [-m, 2 * m, 2 * m]] # rows ALLD, TFT, ALLC
def f(x):
p = [sum(A[i][j] * x[j] for j in range(3)) for i in range(3)]
av = sum(x[i] * p[i] for i in range(3))
return [x[i] * (p[i] - av) for i in range(3)]
won = tot = 0
for i in range(1, n):
for j in range(1, n - i):
x = [i / n, j / n, 1 - (i + j) / n]
for _ in range(int(t / dt)):
d = f(x); x = [max(0.0, x[k] + dt * d[k]) for k in range(3)]
s = sum(x); x = [v / s for v in x]
if max(abs(v) for v in d) < 1e-6: break
tot += 1; won += x[0] > 0.99
return won / tot
print("repeated game, share of starting mixes that end at ALLD (grid of 741 starts):")
for m in (1.2, 2, 5, 10):
print(f" expected rounds m = {m:>4}: {basin(m):.2f}")
# prisoner's dilemma (C, D)
# rest point x = 0.000: stable, ESS
# rest point x = 1.000: unstable
# start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: 0.00 0.00 0.00 0.00 0.00 0.00
# hawk-dove (H, D)
# rest point x = 0.000: unstable
# rest point x = 0.500: stable, ESS
# rest point x = 1.000: unstable
# start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: 0.50 0.50 0.50 0.50 0.50 0.50
# stag hunt (S, H)
# rest point x = 0.000: stable, ESS
# rest point x = 0.750: unstable
# rest point x = 1.000: stable, ESS
# start 0.05 0.25 0.50 0.74 0.76 0.95 -> after t = 40: 0.00 0.00 0.00 0.00 1.00 1.00
# repeated game, share of starting mixes that end at ALLD (grid of 741 starts):
# expected rounds m = 1.2: 1.00
# expected rounds m = 2: 0.74
# expected rounds m = 5: 0.20
# expected rounds m = 10: 0.10The prisoner’s dilemma has a single stable rest point, : everybody defects. Whether we start from 5 % or 95 % cooperators, cooperators have vanished by . The reason is written in the equation: in the donation game a cooperator earns less than a defector whoever the partner is, so is always negative. This is what “in one-shot anonymous interactions defectors eliminate cooperators” means: with no repetition, no reputation and no structure, cooperation earns no return.
But the conclusion holds only for the prisoner’s dilemma. In the hawk–dove game both end points are unstable and the population settles at (with the injury cost): when hawks are many they injure each other; when they are few they take advantage. This mixed state is also an ESS. In the stag hunt both end points are ESSs and the interior point is unstable: if fewer than 75 % hunt stag at the start, everyone ends up hunting hare; if more, everyone ends up hunting stag. Starting from 0.74 and from 0.76 leads to opposite outcomes. The stag hunt is often used to illustrate another difficulty of cooperation: cooperation itself is stable, and the hard part is getting from “nobody cooperates” to “everybody cooperates”.
The last part of the program anticipates Chapter 3. If the same donation game is repeated, with probability of another round after each round, the expected number of rounds is . In a population of always defect (ALLD), tit for tat (TFT) and always cooperate (ALLC), every starting state goes to ALLD when (, below ); when , 74 % of starting states still go to ALLD; when and this share falls to 20 % and 10 %. Repeated interaction does not abolish the basin of attraction of defection; it shrinks it. A population made up mostly of defectors still stays with defection.
Note
Assumptions of the replicator equation. The population is infinite; individuals meet at random; payoffs translate directly into growth rates; strategies are inherited or imitated exactly, without mutation. Each assumption can be relaxed, and relaxing it may change the conclusions. When “replication” is interpreted as imitation learning in people, the equation describes “practices with higher payoffs being adopted by more people”; this is a model, and people do not always imitate those who earn most.
Finite populations: drift and fixation probabilities
Real populations are finite. In a finite population a cooperator mutant has some chance of taking over the whole population by chance (genetic drift), even if cooperators lose out at every step. The quantity that measures this is the fixation probability , the probability that a single mutant eventually replaces everyone. A neutral mutant has fixation probability ; means that selection favours the mutant and that selection opposes it (Nowak et al. 2004).
Calculation
# Finite populations: the Moran process. N individuals; each step one is chosen to
# reproduce with probability proportional to fitness 1 - s + s * (average payoff from
# games with all others), and its offspring replaces a randomly chosen individual. rho =
# probability that a single mutant takes over (neutral benchmark 1/N). Closed form (e.g.
# Nowak et al. 2004):
# rho = 1 / (1 + sum_{k=1}^{N-1} prod_{j=1}^{k} g_j / f_j),
# f_j, g_j = fitness of mutant and resident when there are j mutants.
import random
rng = random.Random(2)
def fit(j, N, g, s):
a, b, c, d = g # mutant row first: a, b; resident row: c, d
pm = (a * (j - 1) + b * (N - j)) / (N - 1)
pr = (c * j + d * (N - j - 1)) / (N - 1)
return 1 - s + s * pm, 1 - s + s * pr
def rho(N, g, s):
tot, prod = 1.0, 1.0
for k in range(1, N):
fm, fr = fit(k, N, g, s); prod *= fr / fm; tot += prod
return 1 / tot
def simulate(N, g, s, runs):
won = 0
for _ in range(runs):
j = 1
while 0 < j < N:
fm, fr = fit(j, N, g, s)
birth_m = rng.random() < j * fm / (j * fm + (N - j) * fr)
death_m = rng.random() < j / N
j += birth_m - death_m
won += j == N
return won / runs
PD = (2, -1, 3, 0) # one-shot anonymous donation game, b = 3, c = 1 (C row first)
DC = (0, 3, -1, 2) # the same game with D as the mutant
print("one-shot donation game, b = 3, c = 1; rho x N (1 = neutral)")
print(" N s cooperator mutant defector mutant")
for N in (10, 50, 200):
for s in (0.01, 0.1):
rc, rd = rho(N, PD, s) * N, rho(N, DC, s) * N
print(f"{N:4d} {s:4.2f} {rc:8.4f} {rd:9.4f}")
print(f"simulation check, N = 20, s = 0.1, 40,000 runs:\n cooperator rho x N ="
f" {simulate(20, PD, 0.1, 40000) * 20:.3f} (formula {rho(20, PD, 0.1) * 20:.3f})")
# Stag hunt, payoffs a, b, c, d = 4, 0, 3, 3 (stag row first): the unstable point is x*
# = (d - b)/(a - b - c + d) = 0.75. The one-third law (Nowak et al. 2004): under weak
# selection in a large population, a single stag hunter is favoured (rho > 1/N) iff x* <
# 1/3.
print("stag hunt, weak selection s = 0.005, N = 100: rho x N for a single stag hunter")
for a in (4, 6, 9, 10, 12, 20):
g = (a, 0, 3, 3); xs = 3 / a
rn = 100 * rho(100, g, 0.005)
print(f" stag payoff {a:>2}: x* = {xs:.3f}, rho x N = {rn:.3f}")
# one-shot donation game, b = 3, c = 1; rho x N (1 = neutral)
# N s cooperator mutant defector mutant
# 10 0.01 0.9408 1.0608
# 10 0.10 0.4941 1.6585
# 50 0.01 0.7611 1.2802
# 50 0.10 0.0246 4.6693
# 200 0.01 0.3070 2.3145
# 200 0.10 0.0000 17.1476
# simulation check, N = 20, s = 0.1, 40,000 runs:
# cooperator rho x N = 0.257 (formula 0.253)
# stag hunt, weak selection s = 0.005, N = 100: rho x N for a single stag hunter
# stag payoff 4: x* = 0.750, rho x N = 0.653
# stag payoff 6: x* = 0.500, rho x N = 0.773
# stag payoff 9: x* = 0.333, rho x N = 0.959
# stag payoff 10: x* = 0.300, rho x N = 1.022
# stag payoff 12: x* = 0.250, rho x N = 1.147
# stag payoff 20: x* = 0.150, rho x N = 1.620The program computes fixation probabilities for the Moran process (at each step one individual is chosen to reproduce in proportion to fitness and its offspring replaces a randomly chosen individual) and reports , the multiple of the neutral value. In the one-shot anonymous donation game a cooperator’s is always below 1 and a defector’s always above 1. The larger the population and the stronger the selection, the larger the gap: with and selection strength a cooperator’s fixation probability is only 6 % below neutral (0.94); with and it is effectively zero, while a defector’s is 17 times the neutral value. A simulation with and (40,000 runs) gives 0.257, in agreement with the formula’s 0.253. In small populations a cooperator occasionally becomes fixed by luck, but the direction of selection is always against it.
Evidence
The one-third law. Nowak, Sasaki, Taylor and Fudenberg showed in 2004 that in large finite populations under weak selection, for a bistable game such as the stag hunt, a single mutant of the cooperative strategy is favoured by selection () if and only if the unstable point is below 1/3. So even when cooperation is stable in the replicator equation, whether it can spread from scratch in a finite population also depends on whether its basin of attraction is large enough (Nowak et al. 2004).
The last part of the program checks this law. With a stag payoff of 4, and ; with a stag payoff of 12, and . The dividing line lies close to : at , , and at it is 1.02. That falls slightly below 1 exactly at 1/3 when is a finite-population correction, which shrinks as the population grows. Chapter 2 shows that the one-third law is one of the three criteria Nowak used to compare the five mechanisms.
Hypothesis · not established
Basin-of-attraction hypothesis (not yet established). The author conjectures that whether a newly founded cooperative group (for instance a newly formed volunteer community) develops a stable norm of mutual help depends mainly on whether the share of founding members “willing to give first” exceeds some threshold, rather than on the average willingness to cooperate; the position of the threshold is set by the payoff structure of the collaboration, much as is in the stag hunt. A testable prediction is that across many newly formed online collaboration groups with identical rules, the relation between the share of active contributors in the first two weeks and the level of contribution after six months is S-shaped (threshold-like) rather than linear. If the relation is linear, or the early share has no predictive power, the hypothesis is refuted.
Chapter summary
The replicator equation makes strategies with above-average payoffs grow. In the one-shot anonymous prisoner’s dilemma defection is better against every partner, and cooperators must disappear; but in the hawk–dove game the two strategies coexist, and in the stag hunt the outcome depends on the starting point. An ESS is a stable point of the replicator dynamics, but not conversely. In finite populations a cooperator can occasionally become fixed by drift, but its fixation probability is below neutral; for bistable games the one-third law gives the condition for spreading from scratch. The next chapter asks what mechanisms can change the payoff structure of the prisoner’s dilemma itself.
References
Fan, Y. (2026). Evolutionary Game Theory, Direct and Indirect Reciprocity and the Emergence of Cooperation: From Hamilton's Rule and Axelrod's Tournaments to Reputation and Costly Punishment, Read Beside Classical Confucian and Mohist Ethics [演化博弈、直接与间接互惠及合作涌现的数学模型] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23788551 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).