5D Library › Evolutionary Game Theory, Direct and Indirect Reciprocity and the Emergence of Cooperation · 中文
Chapter 4 Indirect Reciprocity: Image Scores and Second-Order Reputation Norms
Helping someone you will not meet again
Direct reciprocity needs the same pair to meet repeatedly. In human societies a great deal of help passes between strangers: the helper receives nothing back from the person helped; the return comes from third parties, along the lines of “I helped someone, others saw it, so someone helps me”. This is indirect reciprocity.
Evidence
Image scoring. In 1998 Nowak and Sigmund proposed a model of indirect reciprocity in which every individual has a public “image score” that rises each time it helps someone and falls each time it refuses; “discriminators” help only those whose image score is at least some threshold. Computer simulations and analytical models both showed that such cooperation can be maintained as long as the probability of knowing the other’s image exceeds the cost-to-benefit ratio (Nowak and Sigmund 1998). An experiment with university students by Wedekind and Milinski (2000) supported this: when direct return was impossible, those who had given more in the past received more help (Wedekind and Milinski 2000). Milinski, Semmann and Krambeck (2002) found that alternating rounds of indirect reciprocity with a public goods game kept contributions to the public good high (Milinski et al. 2002).
Image scoring has a flaw. A discriminator who refuses to help a defector with a bad image is, in effect, sanctioning the defector; but in the accounting of image scores, “refusing” means losing points. So the legitimate sanctioner’s own image worsens, other discriminators refuse to help him, his image worsens further … and those who sanction defectors are treated as defectors themselves. Leimar and Hammerstein (2001) found in a more realistic population model that the image-scoring strategy is not stable (Leimar and Hammerstein 2001); Panchanathan and Boyd (2003) pointed out that the problem is that image scoring does not distinguish refusing “without cause” from refusing “with cause”, and that the concept of “standing” proposed by the economist Sugden in 1986 makes exactly this distinction (Panchanathan and Boyd 2003; Sugden 1986).
First-order and second-order assessment
If reputation is written as two labels, “good” and “bad”, assessment rules can be classified by the information they use. A first-order rule looks only at what the donor did: helping is good and refusing is bad; this is binary image scoring. A second-order rule also looks at the recipient’s label.
Calculation
How four common rules assess the donor (G good, B bad):
| Rule | Help a good one | Refuse a good one | Help a bad one | Refuse a bad one |
|---|---|---|---|---|
| Image scoring (first order) | G | B | G | B |
| Shunning | G | B | B | B |
| Simple standing | G | B | G | G |
| Stern judging | G | B | B | G |
The four rules agree in the first two columns; they differ in how they assess dealings with bad individuals. Under simple standing, refusing a bad individual is justified and helping one is not wrong either; under stern judging, refusing a bad individual is justified and helping one is bad; under shunning, any dealing with a bad individual is bad.
Evidence
The leading eight. In 2004 Ohtsuki and Iwasa systematically examined a larger class of rules, in which assessment looks at the donor’s own label, the recipient’s label and the donor’s action (third-order assessment), paired with matching action rules. Among all combinations, only eight allow a population of “discriminators” to maintain a high level of cooperation in the presence of errors and to resist invasion by other strategies; they called them the “leading eight” (Ohtsuki and Iwasa 2004). Their paper of 2006 analysed what the eight norms share: a good individual who helps a good one is assessed as good (maintenance of cooperation); refusing to help a good individual is bad (identification of defectors); a good individual who refuses to help a bad one is good (justified punishment); a bad individual who helps a good one can be assessed as good (forgiveness, or apology). Simple standing and stern judging are both among the leading eight; image scoring and shunning are not (Ohtsuki and Iwasa 2006).
So the framework’s “refusing to help a bad individual costs nothing” holds for all of the leading eight, and it is the key difference from image scoring; whereas “helping a bad individual is judged bad” holds only for some of the norms, such as stern judging, and under simple standing helping a bad individual is still good. The two statements should be kept apart.
A program: reputation dynamics under four rules
The next program simulates a population of 100. At each step a donor and a recipient are drawn at random; helping costs the donor and gives the recipient . Discriminators (DISC) help recipients labelled good and refuse those labelled bad; always cooperate (ALLC) always helps; always defect (ALLD) never helps. A single public observer then gives the donor a new label according to the rule, and everyone adopts that label. There are two kinds of error: intending to help but failing (2 %), and the observer assigning the wrong label (1 %).
Calculation
# Indirect reciprocity with public reputations (everyone shares one view, as in Ohtsuki
# & Iwasa 2004). N = 100 players; each step a random donor meets a random recipient and
# may give b = 3 at cost c = 1. DISC (discriminators) help Good recipients and refuse
# Bad ones; ALLC always help; ALLD never. An observer then labels the DONOR Good or Bad
# by a norm that looks at the action and the recipient's label (second-order norms;
# image scoring looks at the action only). Errors: an intended help fails with prob. e =
# 0.02; the observer assigns the wrong label with prob. mu = 0.01.
# help Good refuse Good help Bad refuse Bad
# image scoring G B G B
# shunning G B B B
# simple standing G B G G
# stern judging G B B G
import random
rng = random.Random(8)
KEYS = (("C", "G"), ("D", "G"), ("C", "B"), ("D", "B"))
NORMS = {name: dict(zip(KEYS, labels)) for name, labels in
(("image scoring", "GBGB"), ("shunning", "GBBB"), ("simple standing", "GBGG"),
("stern judging", "GBBG"))}
B, C, E, MU = 3.0, 1.0, 0.02, 0.01
def run(norm, mix, steps=200000, N=100):
types = [t for t, k in mix.items() for _ in range(k)]
rep = ["G"] * N; pay = [0.0] * N
good = {t: 0 for t in mix}; seen = {t: 0 for t in mix}
zz_help = zz = 0
for step in range(steps):
d, r = rng.sample(range(N), 2)
t = types[d]
want = t == "ALLC" or (t == "DISC" and rep[r] == "G")
act = "C" if want and rng.random() > E else "D"
if act == "C": pay[d] -= C; pay[r] += B
if t == "DISC" and types[r] == "DISC": zz += 1; zz_help += act == "C"
lab = NORMS[norm][(act, rep[r])]
if rng.random() < MU: lab = "B" if lab == "G" else "G"
rep[d] = lab
if step > steps // 4 and step % 50 == 0:
for i in range(N): good[types[i]] += rep[i] == "G"; seen[types[i]] += 1
av = {t: sum(p for p, ty in zip(pay, types) if ty == t) / mix[t] / steps * N
for t in mix}
return {t: good[t] / seen[t] for t in mix}, zz_help / zz, av
print("columns: share Good by strategy | DISC->DISC help rate | payoff per round")
for mix in ({"DISC": 90, "ALLD": 10}, {"DISC": 70, "ALLD": 30}, {"DISC": 80, "ALLC": 20},
{"DISC": 70, "ALLD": 15, "ALLC": 15}):
ks = list(mix)
print("population " + ", ".join(f"{k} {v}" for k, v in mix.items()))
for norm in NORMS:
g, h, av = run(norm, mix)
print(f" {norm:16s} " + " ".join(f"{g[k]:5.2f}" for k in ks)
+ f" | {h:.2f} | " + " ".join(f"{av[k]:+.2f}" for k in ks))
# columns: share Good by strategy | DISC->DISC help rate | payoff per round
# population DISC 90, ALLD 10
# image scoring 0.08 0.01 | 0.08 | +0.14 +0.03
# shunning 0.09 0.01 | 0.09 | +0.16 +0.03
# simple standing 0.97 0.11 | 0.95 | +1.71 +0.28
# stern judging 0.97 0.12 | 0.95 | +1.70 +0.30
# population DISC 70, ALLD 30
# image scoring 0.03 0.01 | 0.04 | +0.05 +0.02
# shunning 0.04 0.01 | 0.04 | +0.05 +0.02
# simple standing 0.98 0.24 | 0.96 | +1.26 +0.51
# stern judging 0.98 0.24 | 0.96 | +1.26 +0.50
# population DISC 80, ALLC 20
# image scoring 0.86 0.97 | 0.84 | +1.73 +1.91
# shunning 0.24 0.25 | 0.25 | +0.94 +0.18
# simple standing 0.97 0.97 | 0.95 | +1.92 +1.90
# stern judging 0.97 0.93 | 0.95 | +1.93 +1.80
# population DISC 70, ALLD 15, ALLC 15
# image scoring 0.45 0.01 0.97 | 0.44 | +0.92 +0.47 +1.44
# shunning 0.06 0.01 0.06 | 0.06 | +0.52 +0.46 -0.43
# simple standing 0.97 0.15 0.97 | 0.95 | +1.61 +0.76 +1.44
# stern judging 0.98 0.17 0.81 | 0.96 | +1.64 +0.80 +1.11The first set of results shows the flaw in image scoring: with 90 % discriminators and 10 % defectors, under image scoring only 8 % of discriminators are rated good, discriminators help each other only 8 % of the time, and their average payoff per round is 0.14, almost as low as the defectors’. Refusing defectors makes discriminators bad, refusing bad discriminators makes more discriminators bad, and cooperation among discriminators collapses. Shunning gives almost the same result. Under simple standing and stern judging 97 % of discriminators are good, they help each other 95 % of the time, and they earn 1.71 and 1.70, far above the defectors’ 0.28 and 0.30. With 30 % defectors the difference is starker.
The image score in the program is the simplest binary version, which remembers only the most recent action; the integer image scores of Nowak and Sigmund’s original paper are somewhat more forgiving. But the flaw runs in the same direction: as long as refusing counts against the refuser, legitimate sanctions are punished.
The third set (80 % discriminators, 20 % unconditional cooperators, no defectors) shows another side. Under image scoring the unconditional cooperators earn more (1.91) than the discriminators (1.73), because discriminators occasionally refuse someone after an error and lose points. Under simple standing the two are almost equal (1.92 and 1.90): when there are no defectors, “unconditional nice guys” and “conditional nice guys” cannot be told apart. Under stern judging the share of unconditional cooperators rated good falls to 93 % and their payoff to 1.80, because they help individuals who have become bad through errors. This shows what stern judging does: it penalises unconditional help to bad individuals, and so prevents unconditional cooperators from slowly increasing in the population, which is the prelude to invasion by defectors.
Calculation
# The model of p08 in an infinite population: the share of Good players of each strategy
# solves a fixed-point equation, payoffs follow, and the replicator equation moves the
# strategy shares (DISC, ALLC, ALLD). Same norms and errors (b = 3, c = 1, e = 0.02, mu
# = 0.01), plus rare mutation: each strategy share drifts towards 1/3 at rate 0.002, so
# a strategy that has died out can come back. A start "ends cooperative" if the final
# state has cooperation above 80 % (share of interactions in which help is given).
NORMS = {"image scoring": {("C", "G"): 1, ("D", "G"): 0, ("C", "B"): 1, ("D", "B"): 0},
"shunning": {("C", "G"): 1, ("D", "G"): 0, ("C", "B"): 0, ("D", "B"): 0},
"simple standing": {("C", "G"): 1, ("D", "G"): 0, ("C", "B"): 1, ("D", "B"): 1},
"stern judging": {("C", "G"): 1, ("D", "G"): 0, ("C", "B"): 0, ("D", "B"): 1}}
B, C, E, MU = 3.0, 1.0, 0.02, 0.01
def reputations(x, nm, g0=None):
gd, gc, ga = g0 or (1.0, 1.0, 1.0)
for _ in range(500):
g = x[0] * gd + x[1] * gc + x[2] * ga # chance a recipient is Good
def good(pc_g, pc_b): # pc_* = P(help) vs Good / Bad
pg = g * (pc_g * nm[("C", "G")] + (1 - pc_g) * nm[("D", "G")]) \
+ (1 - g) * (pc_b * nm[("C", "B")] + (1 - pc_b) * nm[("D", "B")])
return (1 - MU) * pg + MU * (1 - pg)
new = (good(1 - E, 0.0), good(1 - E, 1 - E), good(0.0, 0.0))
if max(abs(a - b) for a, b in zip(new, (gd, gc, ga))) < 1e-12: break
gd, gc, ga = new
return gd, gc, ga
def payoffs(x, nm, g0=None):
gd, gc, ga = reputations(x, nm, g0)
g = x[0] * gd + x[1] * gc + x[2] * ga
recv = lambda gi: B * (1 - E) * (x[1] + x[0] * gi)
return (recv(gd) - C * (1 - E) * g, recv(gc) - C * (1 - E), recv(ga)), (gd, gc, ga)
def evolve(x, nm, t=1500.0, dt=0.2, m=0.002):
reps = None
for _ in range(int(t / dt)):
p, reps = payoffs(x, nm, reps)
av = sum(a * b for a, b in zip(x, p))
y = [max(0.0, xi + dt * (xi * (pi - av) + m * (1 / 3 - xi)))
for xi, pi in zip(x, p)]
s = sum(y); y = [v / s for v in y]
done = max(abs(a - b) for a, b in zip(x, y)) < 1e-9
x = y
if done: break
coop = (1 - E) * (x[0] * (x[0] * reps[0] + x[1] * reps[1] + x[2] * reps[2]) + x[1])
return x, coop
print("check against p08 (DISC 90 %, ALLD 10 %): share Good of DISC, ALLD")
for name, nm in NORMS.items():
gd, gc, ga = reputations([0.9, 0.0, 0.1], nm)
print(f" {name:16s} {gd:.2f} {ga:.2f}")
print("share of 36 starting mixes that end cooperative;\n end state from DISC 0.6,"
" ALLC 0.2, ALLD 0.2:")
starts = [[i / 10, j / 10, 1 - (i + j) / 10]
for i in range(1, 10) for j in range(1, 10 - i)]
for name, nm in NORMS.items():
ok = 0
for s0 in starts:
x, coop = evolve(s0, nm)
ok += coop > 0.8
x, coop = evolve([0.6, 0.2, 0.2], nm)
print(f" {name:16s} {ok / len(starts):.2f} DISC {x[0]:.2f} ALLC {x[1]:.2f}"
f" ALLD {x[2]:.2f}, cooperation {coop:.2f}")
# check against p08 (DISC 90 %, ALLD 10 %): share Good of DISC, ALLD
# image scoring 0.08 0.01
# shunning 0.08 0.01
# simple standing 0.97 0.12
# stern judging 0.97 0.12
# share of 36 starting mixes that end cooperative;
# end state from DISC 0.6, ALLC 0.2, ALLD 0.2:
# image scoring 0.00 DISC 0.11 ALLC 0.00 ALLD 0.89, cooperation 0.00
# shunning 0.00 DISC 1.00 ALLC 0.00 ALLD 0.00, cooperation 0.24
# simple standing 0.53 DISC 0.98 ALLC 0.02 ALLD 0.00, cooperation 0.95
# stern judging 0.61 DISC 0.99 ALLC 0.01 ALLD 0.00, cooperation 0.95The second program solves the reputation equilibrium of the same model in an infinite population and lets the shares of the three strategies evolve by the replicator equation (with a little mutation). It agrees with the first program: with 90 % discriminators and 10 % defectors, 8 % of discriminators are good under image scoring and 97 % under simple standing and stern judging. Of 36 starting mixes, none ends up sustaining cooperation under image scoring or shunning; under simple standing 53 %, and under stern judging 61 %, of the starting mixes go to a state dominated by discriminators with 95 % cooperation. Starting from “discriminators 0.6, unconditional cooperators 0.2, defectors 0.2”, under image scoring the population ends 89 % defectors; under shunning discriminators take over the population, but because errors keep producing “bad” individuals, cooperation is only 24 %; under simple standing and stern judging cooperation is 95 %.
Note
The assumption of public assessment. The results on the leading eight rest on a strong assumption: everyone agrees on who is good and who is bad, and assessments are public and shared. Hilbe, Schmid, Tkadlec, Chatterjee and Nowak (2018) found that when each individual observes privately and assesses independently, with incomplete or noisy observations, opinions quickly diverge between individuals and most of the leading eight can no longer sustain cooperation (Hilbe et al. 2018). So “adopting a second-order assessment rule” is not enough on its own to guarantee cooperation; how assessment information is gathered and shared matters just as much. The checklist in Chapter 8 lists this as a separate item.
“Repay resentment with uprightness”: an analogy, not an identity
There is an evocative parallel between the norms of indirect reciprocity and classical ethics. In the Analects (“Xian Wen”), someone asks: “What about repaying resentment with kindness?” Confucius replies: “Then with what would you repay kindness? Repay resentment with uprightness, and repay kindness with kindness.” (Chapter 7 gives the full text.) “Repay kindness with kindness” resembles “a good individual helping a good one is good”; “repay resentment with uprightness” bears some resemblance to “refusing a bad individual is good”: towards someone who has wronged you, you need not return kindness, but you also do not return resentment; you deal with him uprightly.
The parallel is only structural. Confucius is discussing how a gentleman should personally regard favours and grievances; in Zhu Xi’s commentary “uprightness” means “in what one loves and hates, takes and rejects, being entirely impartial and without selfishness”, a fair state of mind, not an action rule of “refusing to help”; reputation norms concern an assessment system enforced by third parties in a group, whose effects are measured by changes in strategy frequencies. To say that Confucius “had already proposed second-order reputation norms” misunderstands both Confucius and the models.
Hypothesis · not established
Justified-refusal labelling hypothesis (not yet established). The author conjectures that in an online community built on mutual help, if the public contribution record distinguishes “refusing help to a member flagged for breaking the rules” from “refusing without cause”, members’ willingness to sanction rule-breakers will rise without reducing the help given to members in good standing. A testable prediction is that, in communities randomly assigned to the two kinds of record, rule-breakers in communities that distinguish justified refusals receive less help after being flagged, while mutual help among members in good standing is no lower than in control communities. If the two kinds of community do not differ in help given to rule-breakers, or distinguishing justified refusals reduces mutual help among members in good standing, the hypothesis is refuted. The trial in Chapter 8 tests a related question. Its labels follow a single rule, simple standing from the table above with the strictest form of justification: withholding from someone in bad standing is good, withholding from someone in good standing is bad, and contributing to someone in bad standing costs nothing; in a group of three, withholding counts as justified only when both other members are bad. It is not stern judging (helping a bad individual is not penalised), nor image scoring (which does not distinguish justified from unjustified withholding).
Chapter summary
Indirect reciprocity allows cooperation between strangers who will not meet again, provided reputations are known well enough (). First-order image scoring has a flaw: refusing to help a bad individual also costs points, so legitimate sanctioners are treated as defectors and cooperation collapses when defectors are present. Second-order assessment distinguishes “refusing without cause” from “justified refusal”; all of Ohtsuki and Iwasa’s leading eight norms have this property, and stern judging also rates helping a bad individual as bad, which stops unconditional cooperators from increasing. These results depend on the assumption that assessments are public and shared. The next chapter discusses a mechanism that needs no reputation: who is next to whom.
References
Fan, Y. (2026). Evolutionary Game Theory, Direct and Indirect Reciprocity and the Emergence of Cooperation: From Hamilton's Rule and Axelrod's Tournaments to Reputation and Costly Punishment, Read Beside Classical Confucian and Mohist Ethics [演化博弈、直接与间接互惠及合作涌现的数学模型] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23788551 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).