What does Mid-Context Forgetting and Relay Thresholds conclude, in numbers, about “The Product of Retention Rates: a Ceiling on Layers, the Recency Slope of Running Summaries and the Casualties among Details”?

Answer: Chapter 4 computes the product of relay retention rates, shows that a running summary turns the U into a recency slope, and finds detailed facts keeping only 0.4305 after eight layers.

Scope of the numbers: Tasks, material and parameters are illustrative and the model constants come from one 2023 model; all calculations are generated by the book's programs, which call no model. It is not individual business advice and neither rates nor recommends any model or service.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.4305

mean per-layer retention E r = 0.9475
L(E r)^LE[r^L]Monte Carlodetails onlysalient only
10.94750.94750.94690.90000.9950
20.89780.90000.89900.81000.9900
40.80600.81810.81950.65610.9801
80.64960.69560.69530.43050.9607
160.42200.55410.55360.18530.9229
after 8 layers details are 0.309 of the surviving facts

Key formula

\[R_L = r^L .\]
Show the full program
# Chapter 4. p43: when facts differ, the average hides the casualties.
# Illustrative mixture: half the facts are salient (kept with chance
# 0.995 per layer), half are details such as figures, names and dates
# (kept with chance 0.90). A summary is scored by its average r, but
# survival after L layers is E[r^L], which by Jensen's inequality is
# at least (E r)^L, while the detail class alone decays as 0.90^L.
# Fixed-seed Monte Carlo of 20000 facts checks the exact numbers.
import math
import random

CLASSES = ((0.5, 0.995), (0.5, 0.90))
FACTS = 20000
rng = random.Random(20261113)

mean_r = math.fsum(w * r for w, r in CLASSES)
print("mean per-layer retention E r = %.4f" % mean_r)
print("  L  (E r)^L   E[r^L]  Monte Carlo  details only  salient only")
for depth in (1, 2, 4, 8, 16):
    exact = math.fsum(w * r ** depth for w, r in CLASSES)
    alive = 0
    for j in range(FACTS):
        r = CLASSES[0][1] if j % 2 == 0 else CLASSES[1][1]
        if all(rng.random() < r for _ in range(depth)):
            alive += 1
    print("%3d  %7.4f  %7.4f  %11.4f  %12.4f  %12.4f"
          % (depth, mean_r ** depth, exact, alive / FACTS,
             CLASSES[1][1] ** depth, CLASSES[0][1] ** depth))
share = 0.5 * 0.90 ** 8 / math.fsum(w * r ** 8 for w, r in CLASSES)
print("after 8 layers details are %.3f of the surviving facts" % share)

Single source and citation

Fan, Y. (2026). Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays (Version 1.0.0), Chapter 4: The Product of Retention Rates: a Ceiling on Layers, the Recency Slope of Running Summaries and the Casualties among Details. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-255 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026255,
  author    = {Fan, Yafu},
  title     = {Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book