What does Mid-Context Forgetting and Relay Thresholds conclude, in numbers, about “The Product of Retention Rates: a Ceiling on Layers, the Recency Slope of Running Summaries and the Casualties among Details”?
Answer: Chapter 4 computes the product of relay retention rates, shows that a running summary turns the U into a recency slope, and finds detailed facts keeping only 0.4305 after eight layers.
Scope of the numbers: Tasks, material and parameters are illustrative and the model constants come from one 2023 model; all calculations are generated by the book's programs, which call no model. It is not individual business advice and neither rates nor recommends any model or service.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.4305
| mean per-layer retention E r = 0.9475 | |||||
| L | (E r)^L | E[r^L] | Monte Carlo | details only | salient only |
| 1 | 0.9475 | 0.9475 | 0.9469 | 0.9000 | 0.9950 |
| 2 | 0.8978 | 0.9000 | 0.8990 | 0.8100 | 0.9900 |
| 4 | 0.8060 | 0.8181 | 0.8195 | 0.6561 | 0.9801 |
| 8 | 0.6496 | 0.6956 | 0.6953 | 0.4305 | 0.9607 |
| 16 | 0.4220 | 0.5541 | 0.5536 | 0.1853 | 0.9229 |
| after 8 layers details are 0.309 of the surviving facts |
Key formula
Show the full program
# Chapter 4. p43: when facts differ, the average hides the casualties.
# Illustrative mixture: half the facts are salient (kept with chance
# 0.995 per layer), half are details such as figures, names and dates
# (kept with chance 0.90). A summary is scored by its average r, but
# survival after L layers is E[r^L], which by Jensen's inequality is
# at least (E r)^L, while the detail class alone decays as 0.90^L.
# Fixed-seed Monte Carlo of 20000 facts checks the exact numbers.
import math
import random
CLASSES = ((0.5, 0.995), (0.5, 0.90))
FACTS = 20000
rng = random.Random(20261113)
mean_r = math.fsum(w * r for w, r in CLASSES)
print("mean per-layer retention E r = %.4f" % mean_r)
print(" L (E r)^L E[r^L] Monte Carlo details only salient only")
for depth in (1, 2, 4, 8, 16):
exact = math.fsum(w * r ** depth for w, r in CLASSES)
alive = 0
for j in range(FACTS):
r = CLASSES[0][1] if j % 2 == 0 else CLASSES[1][1]
if all(rng.random() < r for _ in range(depth)):
alive += 1
print("%3d %7.4f %7.4f %11.4f %12.4f %12.4f"
% (depth, mean_r ** depth, exact, alive / FACTS,
CLASSES[1][1] ** depth, CLASSES[0][1] ** depth))
share = 0.5 * 0.90 ** 8 / math.fsum(w * r ** 8 for w, r in CLASSES)
print("after 8 layers details are %.3f of the surviving facts" % share)Single source and citation
Fan, Y. (2026). Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays (Version 1.0.0), Chapter 4: The Product of Retention Rates: a Ceiling on Layers, the Recency Slope of Running Summaries and the Casualties among Details. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-255 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026255,
author = {Fan, Yafu},
title = {Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Collapse in the Middle: Original Numbers, Effect Sizes and Dose for the U-Shaped Position Curve
- Chapter 2 · Dissent and Diversity: Four Curves, Bigger Windows and the Single-Needle Test
- Chapter 3 · A Position-Recall Model: a Primacy Bump, a Recency Bump and a Floor That Sinks with Length
- Chapter 5 · The Best Segment Length and the Relay Threshold: a Tug of War between Losses in Reading and in Writing
- Chapter 6 · The Key-Fact Restatement Slot: Zero Loss Comes Only from Template Copying and Checksums
- Chapter 7 · The Classics Examined: Mencius on One Day of Sun and Ten Days of Cold, and Ebbinghaus on Memory
- Chapter 8 · Closing the Loop: a Relay Template for Long Tasks, the Key-Fact Slot and a Pre-Registered Test