Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays: chapter answers

Two views circulate. One says that whatever fits in the window gets read, so long material can simply be stuffed in; the other says that models reading long inputs forget the middle, so material should be cut into segments, summarised and relayed layer by layer. This book tests the founding claim of the second view: recall in the middle of a long context is markedly lower than at the ends, tracing a U (Liu et al., Transactions of the Association for Computational Linguistics, 2024), so a relay beats stuffing the window, while relay depth is bounded by the product of per-layer retention rates.…

DOI pending · monograph page goes live after publication · 5DL-B-255 · 中文版

Illustrative model calculation, not a live quote