What does Mid-Context Forgetting and Relay Thresholds conclude, in numbers, about “The Collapse in the Middle: Original Numbers, Effect Sizes and Dose for the U-Shaped Position Curve”?

Answer: Program p11 fits GPT-3.5-Turbo's scores at twenty documents with an upward-opening parabola 17.09 points deep. Program p12 gives the confidence interval of the 0.220 gap between beginning and middle, shows that "below closed-book" straddles zero, and splits the loss at the worst position into position and dilution. Program p13 shows that more documents push the worst position lower and the mean capture from 0.741 down to 0.626. All of these numbers come from one model on one task. Does another model still draw a U? And does a version with a larger window read any better?

Scope of the numbers: Tasks, material and parameters are illustrative and the model constants come from one 2023 model; all calculations are generated by the book's programs, which call no model. It is not individual business advice and neither rates nor recommends any model or service.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.626, 0.741

docs first min last primacy recency mean mean/oracle
1076.8 61.2 62.4 15.6 1.2 65.46 0.741
2075.8 53.8 63.2 22.0 9.4 59.12 0.670
3073.4 50.5 63.7 22.9 13.2 55.43 0.626
worst case61.2 -> 53.8 -> 50.5
each 10 more documents cost the worst position 5.4 points
Show the full program
# Chapter 1. p13: dose-response of the middle dip to context length.
# Data: Liu et al. (2024; arXiv v3 appendix Tables 5 to 7), GPT-3.5-
# Turbo with 10 and 20 documents, GPT-3.5-Turbo (16K) with 30
# documents (about 2K, 4K and 6K tokens). Accuracy in per cent at the
# reported positions. Oracle (one document only): 88.3 and 88.6.
# The mean over all positions is read off a piecewise-linear curve
# through the reported points, a crude stand-in for a fact whose
# position is unknown in advance.
import math

RUNS = (
    (10, (1, 5, 10), (76.8, 61.2, 62.4), 88.3),
    (20, (1, 5, 10, 15, 20), (75.8, 57.2, 53.8, 55.4, 63.2), 88.3),
    (30, (1, 5, 10, 15, 20, 25, 30),
     (73.4, 55.1, 50.5, 50.9, 51.8, 54.9, 63.7), 88.6),
)


def interp(xs, ys, x):
    for (x0, y0), (x1, y1) in zip(zip(xs, ys), zip(xs[1:], ys[1:])):
        if x0 <= x <= x1:
            return y0 + (y1 - y0) * (x - x0) / (x1 - x0)
    raise ValueError(x)


print("docs  first   min  last  primacy recency  mean  mean/oracle")
for n, xs, ys, orc in RUNS:
    lo = min(ys)
    mean = math.fsum(interp(xs, ys, k) for k in range(1, n + 1)) / n
    print("%4d  %5.1f %5.1f %5.1f   %5.1f   %5.1f   %5.2f   %6.3f"
          % (n, ys[0], lo, ys[-1], ys[0] - lo, ys[-1] - lo, mean,
             mean / orc))
worst = [min(r[2]) for r in RUNS]
print("worst case: " + " -> ".join("%.1f" % w for w in worst))
print("each 10 more documents cost the worst position %.1f points"
      % ((worst[0] - worst[-1]) / 2))

Single source and citation

Fan, Y. (2026). Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays (Version 1.0.0), Chapter 1: The Collapse in the Middle: Original Numbers, Effect Sizes and Dose for the U-Shaped Position Curve. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-255 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026255,
  author    = {Fan, Yafu},
  title     = {Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book