What does Mid-Context Forgetting and Relay Thresholds conclude, in numbers, about “The Best Segment Length and the Relay Threshold: a Tug of War between Losses in Reading and in Writing”?

Answer: Chapter 5 solves for the best segment length and the relay threshold: for the 2023 reader each summary call must keep key facts above about 0.911 and segments must be short, and once the reader's loss falls below 0.413 of the original curve a relay at fidelity 0.97 loses to the whole window.

Scope of the numbers: Tasks, material and parameters are illustrative and the model constants come from one 2023 model; all calculations are generated by the book's programs, which call no model. It is not individual business advice and neither rates nor recommends any model or service.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.413, 0.911, 0.97

lamstuffedrelay q=.99q=.97q=.95winner at q=.97
1.000.4760.9230.7840.663relay
0.500.7380.9230.7840.674relay
0.250.8690.9230.8060.764stuffed
0.100.9480.9260.8820.846stuffed
0.050.9740.9490.9110.874stuffed
0.001.0000.9800.9410.902stuffed
q = 0.99: relay wins only when lam exceeds 0.148
q = 0.97: relay wins only when lam exceeds 0.413
q = 0.95: relay wins only when lam exceeds 0.643

Key formula

\[R(s) = \big(q\, c(s)\big)^{L},\]
Show the full program
# Chapter 5. p53: newer, flatter readers erase the relay's advantage.
# Later work flattens the curve (calibration, position-aware training,
# near-perfect single-needle scores). We shrink every loss of the p32
# reader by a factor lam: rho_lam = 1 - lam (1 - rho). lam = 1 is the
# 2023 GPT-3.5-Turbo shape, lam = 0 a perfectly flat reader. For
# D = 256 units and per-call fidelity q we compare the best tree relay
# (segment s from 2 to 64) with one stuffed window. Illustrative.
import math

TAU, M0, K, A, B = 2.4, 0.752, 0.0505, 0.251, 0.128
D = 256


def rho(i, n, lam):
    v = (M0 - K * math.log(n) + A * math.exp(-(i - 1) / TAU)
         + B * math.exp(-(n - i) / TAU))
    return 1.0 - lam * (1.0 - min(1.0, v))


def capture(n, lam):
    return math.fsum(rho(i, n, lam) for i in range(1, n + 1)) / n


def layers(n, f):
    depth, width = 0, n
    while width > 1:
        width = -(-width // f)
        depth += 1
    return max(depth, 1)


def best_relay(q, lam):
    cap = {s: capture(s, lam) for s in range(2, 65)}
    return max((q * cap[s]) ** layers(D, s) for s in cap)


print("  lam   stuffed   relay q=.99  q=.97  q=.95   winner at q=.97")
for lam in (1.0, 0.5, 0.25, 0.1, 0.05, 0.0):
    st = capture(D, lam)
    rs = [best_relay(q, lam) for q in (0.99, 0.97, 0.95)]
    print("%5.2f   %7.3f   %10.3f %6.3f %6.3f   %s"
          % (lam, st, rs[0], rs[1], rs[2],
             "relay" if rs[1] > st else "stuffed"))
for q in (0.99, 0.97, 0.95):
    lo, hi = 0.0, 1.0
    for _ in range(40):
        mid = (lo + hi) / 2
        if best_relay(q, mid) > capture(D, mid):
            hi = mid
        else:
            lo = mid
    print("q = %.2f: relay wins only when lam exceeds %.3f" % (q, hi))

Single source and citation

Fan, Y. (2026). Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays (Version 1.0.0), Chapter 5: The Best Segment Length and the Relay Threshold: a Tug of War between Losses in Reading and in Writing. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-255 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026255,
  author    = {Fan, Yafu},
  title     = {Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book