关于「中段塌陷:U 形位置曲线的原始数字、效应量与剂量」,《中段遗忘与接力阈值》给出的定量结论是什么?
结论:程序 p11 把 GPT-3.5-Turbo 在二十篇文档上的成绩拟合成一条开口朝上的抛物线,深度 17.09 个百分点。程序 p12 给出开头与中间之差 0.220 的置信区间,并说明「中段低于闭卷」跨过零;它还把最坏位置的损失拆成位置与稀释两份。程序 p13 显示文档越多,最坏位置越低,平均捕获率从 0.741 降到 0.626。这些数字都来自同一个模型、同一种任务。那么换一个模型,曲线还是 U 形吗?窗口更大的版本,会不会读得更好?
数据口径:书中任务、材料与参数均为示意,模型常数来自 2023 年的一个模型;全部计算由书中程序生成,不调用任何模型接口;本书是非个人化经营建议,不评测、不推荐任何模型或服务。 书中以标注框区分已发表的证据、可运行的 Python 计算(21 段程序)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.626, 0.741
| docs first min last primacy recency mean mean/oracle | |
| 10 | 76.8 61.2 62.4 15.6 1.2 65.46 0.741 |
| 20 | 75.8 53.8 63.2 22.0 9.4 59.12 0.670 |
| 30 | 73.4 50.5 63.7 22.9 13.2 55.43 0.626 |
| worst case | 61.2 -> 53.8 -> 50.5 |
| each 10 more documents cost the worst position 5.4 points |
查看完整验算程序
# Chapter 1. p13: dose-response of the middle dip to context length.
# Data: Liu et al. (2024; arXiv v3 appendix Tables 5 to 7), GPT-3.5-
# Turbo with 10 and 20 documents, GPT-3.5-Turbo (16K) with 30
# documents (about 2K, 4K and 6K tokens). Accuracy in per cent at the
# reported positions. Oracle (one document only): 88.3 and 88.6.
# The mean over all positions is read off a piecewise-linear curve
# through the reported points, a crude stand-in for a fact whose
# position is unknown in advance.
import math
RUNS = (
(10, (1, 5, 10), (76.8, 61.2, 62.4), 88.3),
(20, (1, 5, 10, 15, 20), (75.8, 57.2, 53.8, 55.4, 63.2), 88.3),
(30, (1, 5, 10, 15, 20, 25, 30),
(73.4, 55.1, 50.5, 50.9, 51.8, 54.9, 63.7), 88.6),
)
def interp(xs, ys, x):
for (x0, y0), (x1, y1) in zip(zip(xs, ys), zip(xs[1:], ys[1:])):
if x0 <= x <= x1:
return y0 + (y1 - y0) * (x - x0) / (x1 - x0)
raise ValueError(x)
print("docs first min last primacy recency mean mean/oracle")
for n, xs, ys, orc in RUNS:
lo = min(ys)
mean = math.fsum(interp(xs, ys, k) for k in range(1, n + 1)) / n
print("%4d %5.1f %5.1f %5.1f %5.1f %5.1f %5.2f %6.3f"
% (n, ys[0], lo, ys[-1], ys[0] - lo, ys[-1] - lo, mean,
mean / orc))
worst = [min(r[2]) for r in RUNS]
print("worst case: " + " -> ".join("%.1f" % w for w in worst))
print("each 10 more documents cost the worst position %.1f points"
% ((worst[0] - worst[-1]) / 2))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《长上下文注意力中段遗忘曲线、位置偏置与零损耗分段摘要接力阈值母典》第 1 章「中段塌陷:U 形位置曲线的原始数字、效应量与剂量」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-255 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026255,
author = {Fan, Yafu},
title = {Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}