What does Costly Signals and Hot-Cold Traps conclude, in numbers, about “Reward Prediction Errors: a Small Temporal-Difference Model of the Hot-and-Cold Relationship”?

Answer: By mean warmth the order is S1, S2, S3, S4: 0.800, 0.400, 0.372 and 0.244. By rush the order is almost exactly reversed. S1 and S2 both score 0.000, S3 scores 0.245 and S4 0.193, and the largest single rushes belong to S3 (0.917) and S4 (0.887). A learner who judged by how intense things feel would pick the two partners who give least; one who judged by mean warmth would pick S1.

Scope of the numbers: Its parameters are illustrative; all calculations are generated by its programs; it is not psychological counselling, claims no effect of any kind, and teaches no technique of manipulation. Chapter 2 enters each interaction in a likelihood-ratio ledger; with illustrative parameters one missed plan weighs as much as 2.52 kept ones, and Wald's sequential test saves 0.459 of the interactions a fixed observation period needs.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.000, 0.193, 0.244, 0.245, 0.372, 0.400

partner mean reward rush/round biggest rush leaves at
S1 steady 0.80.800 0.000 0.009 stays
S2 steady 0.40.400 0.000 0.005 round 21
S3 random 0.40.372 0.245 0.917 round 25
S4 flood+cycle0.244 0.193 0.887 round 29
rush ranks S3 and S4 on top; mean reward ranks them last
Show the full program
# Chapter 4. p42: felt intensity is not value. A delta-rule learner
# (learning rate 0.2, start 0) meets four partners for 200 rounds:
# S1 steady warm 0.8 each round; S2 steady lukewarm 0.4; S3 random,
# warm (1) with probability 0.4; S4 a 20-round flood of warmth, then a
# cycle of 6 cold rounds and 2 warm ones. "Rush" = mean positive error
# per round, rounds 21 to 200. The average rule leaves once the mean
# reward of the last 12 rounds falls below 0.5 (checked from round 21).
# Illustrative values; S3 uses a fixed seed.
import math
import random

ALPHA, N, WIN, FLOOR = 0.2, 200, 12, 0.5
rng = random.Random(20261112)


def s1(t):
    return 0.8


def s2(t):
    return 0.4


def s3(t):
    return 1.0 if rng.random() < 0.4 else 0.0


def s4(t):
    if t < 20:
        return 1.0
    return 1.0 if (t - 20) % 8 >= 6 else 0.0


print("partner          mean reward  rush/round  biggest rush  leaves at")
for lab, f in (("S1 steady 0.8", s1), ("S2 steady 0.4", s2),
               ("S3 random 0.4", s3), ("S4 flood+cycle", s4)):
    v, rs, pos, top, leave = 0.0, [], [], 0.0, None
    for t in range(N):
        r = f(t)
        d = r - v
        v += ALPHA * d
        rs.append(r)
        if t >= 20:
            pos.append(max(d, 0.0))
            top = max(top, d)
            recent = math.fsum(rs[-WIN:]) / WIN
            if leave is None and recent < FLOOR:
                leave = t + 1
    mr = math.fsum(rs[20:]) / (N - 20)
    lv = "round {:3d}".format(leave) if leave else "  stays"
    print("{:15s}  {:11.3f}  {:10.3f}  {:12.3f}  {}".format(
        lab, mr, math.fsum(pos) / len(pos), top, lv))
print("rush ranks S3 and S4 on top; mean reward ranks them last")

Single source and citation

Fan, Y. (2026). Costly-Signal Screening in Early Intimacy, Intermittent-Reinforcement Dopamine Traps and Reciprocal-Floor Stopping Games (Version 1.0.0), Chapter 4: Reward Prediction Errors: a Small Temporal-Difference Model of the Hot-and-Cold Relationship. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-181 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026181,
  author    = {Fan, Yafu},
  title     = {Costly-Signal Screening in Early Intimacy, Intermittent-Reinforcement Dopamine Traps and Reciprocal-Floor Stopping Games},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book