What does Costly Signals and Hot-Cold Traps conclude, in numbers, about “Reward Prediction Errors: a Small Temporal-Difference Model of the Hot-and-Cold Relationship”?
Answer: By mean warmth the order is S1, S2, S3, S4: 0.800, 0.400, 0.372 and 0.244. By rush the order is almost exactly reversed. S1 and S2 both score 0.000, S3 scores 0.245 and S4 0.193, and the largest single rushes belong to S3 (0.917) and S4 (0.887). A learner who judged by how intense things feel would pick the two partners who give least; one who judged by mean warmth would pick S1.
Scope of the numbers: Its parameters are illustrative; all calculations are generated by its programs; it is not psychological counselling, claims no effect of any kind, and teaches no technique of manipulation. Chapter 2 enters each interaction in a likelihood-ratio ledger; with illustrative parameters one missed plan weighs as much as 2.52 kept ones, and Wald's sequential test saves 0.459 of the interactions a fixed observation period needs.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.000, 0.193, 0.244, 0.245, 0.372, 0.400
| partner mean reward rush/round biggest rush leaves at | |
| S1 steady 0.8 | 0.800 0.000 0.009 stays |
| S2 steady 0.4 | 0.400 0.000 0.005 round 21 |
| S3 random 0.4 | 0.372 0.245 0.917 round 25 |
| S4 flood+cycle | 0.244 0.193 0.887 round 29 |
| rush ranks S3 and S4 on top; mean reward ranks them last |
Show the full program
# Chapter 4. p42: felt intensity is not value. A delta-rule learner
# (learning rate 0.2, start 0) meets four partners for 200 rounds:
# S1 steady warm 0.8 each round; S2 steady lukewarm 0.4; S3 random,
# warm (1) with probability 0.4; S4 a 20-round flood of warmth, then a
# cycle of 6 cold rounds and 2 warm ones. "Rush" = mean positive error
# per round, rounds 21 to 200. The average rule leaves once the mean
# reward of the last 12 rounds falls below 0.5 (checked from round 21).
# Illustrative values; S3 uses a fixed seed.
import math
import random
ALPHA, N, WIN, FLOOR = 0.2, 200, 12, 0.5
rng = random.Random(20261112)
def s1(t):
return 0.8
def s2(t):
return 0.4
def s3(t):
return 1.0 if rng.random() < 0.4 else 0.0
def s4(t):
if t < 20:
return 1.0
return 1.0 if (t - 20) % 8 >= 6 else 0.0
print("partner mean reward rush/round biggest rush leaves at")
for lab, f in (("S1 steady 0.8", s1), ("S2 steady 0.4", s2),
("S3 random 0.4", s3), ("S4 flood+cycle", s4)):
v, rs, pos, top, leave = 0.0, [], [], 0.0, None
for t in range(N):
r = f(t)
d = r - v
v += ALPHA * d
rs.append(r)
if t >= 20:
pos.append(max(d, 0.0))
top = max(top, d)
recent = math.fsum(rs[-WIN:]) / WIN
if leave is None and recent < FLOOR:
leave = t + 1
mr = math.fsum(rs[20:]) / (N - 20)
lv = "round {:3d}".format(leave) if leave else " stays"
print("{:15s} {:11.3f} {:10.3f} {:12.3f} {}".format(
lab, mr, math.fsum(pos) / len(pos), top, lv))
print("rush ranks S3 and S4 on top; mean reward ranks them last")Single source and citation
Fan, Y. (2026). Costly-Signal Screening in Early Intimacy, Intermittent-Reinforcement Dopamine Traps and Reciprocal-Floor Stopping Games (Version 1.0.0), Chapter 4: Reward Prediction Errors: a Small Temporal-Difference Model of the Hot-and-Cold Relationship. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-181 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026181,
author = {Fan, Yafu},
title = {Costly-Signal Screening in Early Intimacy, Intermittent-Reinforcement Dopamine Traps and Reciprocal-Floor Stopping Games},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · When Costly Signals Separate: Single Crossing, the Handicap Principle and the Cost Structure of Honest Signals
- Chapter 2 · Sequential Bayesian Screening: a Likelihood-Ratio Ledger and Wald's Boundaries for Going On or Stepping Back
- Chapter 3 · Variable-Ratio Reinforcement and Resistance to Extinction: Why Responses That Come and Go Are the Hardest to Give Up
- Chapter 5 · Optimal Stopping: the 1/e of the Secretary Problem, Thresholds on a Known Scale and the Reciprocal-Floor Rule
- Chapter 6 · Detectable Fingerprints of Manipulation: Features and False Alarm Rates for the Love-Bombing and Withdrawal Cycle
- Chapter 7 · Reading the Sources: the Six Rites of the Book of Rites and Zahavi's Handicap Principle
- Chapter 8 · Practice and Refutation: a Personal Interaction Log, a Stop-Loss Communication Kit and a Pre-Registered Test