关于「位置召回模型:首因峰、近因峰与随长度下沉的地板」,《中段遗忘与接力阈值》给出的定量结论是什么?
结论:程序 p31 用双指数模型分别拟合三十篇与二十篇数据,首因峰高度都是 0.249,衰减长度 2.5 与 2.2 篇,只有地板从 0.609 降到 0.578。程序 p32 把这一结构写成联合模型,五个参数拟合十二个格子,误差 0.0105,并外推出捕获曲线:四篇 0.908,三十二篇 0.612,两百五十六篇 0.477。这条曲线的每一个数都可以由经营者用自己的读者重测,第八章会给出方法。有了这条曲线,我们就能问接力的问题了:把长材料切成短段、写摘要、再往下传,每传一层会丢多少,传几层就不划算了?
数据口径:书中任务、材料与参数均为示意,模型常数来自 2023 年的一个模型;全部计算由书中程序生成,不调用任何模型接口;本书是非个人化经营建议,不评测、不推荐任何模型或服务。 书中以标注框区分已发表的证据、可运行的 Python 计算(21 段程序)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0105, 0.477, 0.578, 0.612, 0.908, 32
| joint fit: tau 2.4 | M0 0.752 | K 0.0505 | A 0.251 | B 0.128 | |
| rmse over 12 cells: 0.0105 (relative recall) | |||||
| n | floor | rho(1) | rho(mid) | rho(n) | capture c(n) |
| 1 | 0.752 | 1.000 | 1.000 | 1.000 | 1.000 |
| 2 | 0.717 | 1.000 | 1.000 | 1.000 | 1.000 |
| 4 | 0.683 | 0.971 | 0.904 | 0.883 | 0.908 |
| 8 | 0.648 | 0.906 | 0.744 | 0.789 | 0.782 |
| 16 | 0.613 | 0.864 | 0.631 | 0.741 | 0.682 |
| 32 | 0.578 | 0.829 | 0.578 | 0.706 | 0.612 |
| 64 | 0.543 | 0.794 | 0.543 | 0.671 | 0.560 |
| 128 | 0.508 | 0.759 | 0.508 | 0.636 | 0.516 |
| 256 | 0.473 | 0.724 | 0.473 | 0.601 | 0.477 |
核心公式
\[\rho(i, n) = m + A\, e^{-(i-1)/\tau} + B\, e^{-(n-i)/\tau},\]
查看完整验算程序
# Chapter 3. p32: one model for both lengths, and the capture curve.
# Joint fit (ours) to the 20- and 30-document GPT-3.5-Turbo curves of
# Liu et al. (2024), relative to oracle accuracy:
# rho(i, n) = M0 - K ln n + A exp(-(i-1)/tau) + B exp(-(n-i)/tau),
# capped at 1. The floor falls with log length (dilution); the two
# bumps keep a fixed size in documents. Grid on tau, least squares on
# M0, K, A, B. Capture c(n) = mean of rho over the n positions: the
# expected recall of a fact whose position is not known in advance.
# Extrapolation beyond 30 documents is illustrative only.
import math
DATA = (
(30, (1, 5, 10, 15, 20, 25, 30),
(73.4, 55.1, 50.5, 50.9, 51.8, 54.9, 63.7), 88.6),
(20, (1, 5, 10, 15, 20), (75.8, 57.2, 53.8, 55.4, 63.2), 88.3),
)
def solve(a, b):
k = len(b)
m = [row[:] + [v] for row, v in zip(a, b)]
for c in range(k):
p = max(range(c, k), key=lambda r: abs(m[r][c]))
m[c], m[p] = m[p], m[c]
for r in range(k):
if r != c:
f = m[r][c] / m[c][c]
m[r] = [x - f * y for x, y in zip(m[r], m[c])]
return [m[i][k] / m[i][i] for i in range(k)]
def design(tau):
rows, ys = [], []
for n, xs, acc, orc in DATA:
for x, v in zip(xs, acc):
rows.append((1.0, -math.log(n), math.exp(-(x - 1) / tau),
math.exp(-(n - x) / tau)))
ys.append(v / orc)
return rows, ys
def fit(tau):
rows, ys = design(tau)
ata = [[math.fsum(r[i] * r[j] for r in rows) for j in range(4)]
for i in range(4)]
aty = [math.fsum(r[i] * y for r, y in zip(rows, ys))
for i in range(4)]
par = solve(ata, aty)
res = [y - math.fsum(p * q for p, q in zip(par, r))
for r, y in zip(rows, ys)]
return math.sqrt(math.fsum(e * e for e in res) / len(res)), par
best = min(((fit(t / 10)[0], t / 10) for t in range(5, 151)))
rmse, par = fit(best[1])
m0, k, a, b = par
tau = best[1]
print("joint fit: tau %.1f M0 %.3f K %.4f A %.3f B %.3f"
% (tau, m0, k, a, b))
print("rmse over 12 cells: %.4f (relative recall)" % rmse)
def rho(i, n):
v = (m0 - k * math.log(n) + a * math.exp(-(i - 1) / tau)
+ b * math.exp(-(n - i) / tau))
return min(1.0, v)
print(" n floor rho(1) rho(mid) rho(n) capture c(n)")
for n in (1, 2, 4, 8, 16, 32, 64, 128, 256):
c = math.fsum(rho(i, n) for i in range(1, n + 1)) / n
print("%5d %6.3f %6.3f %8.3f %6.3f %8.3f"
% (n, m0 - k * math.log(n), rho(1, n), rho((n + 1) // 2, n),
rho(n, n), c))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《长上下文注意力中段遗忘曲线、位置偏置与零损耗分段摘要接力阈值母典》第 3 章「位置召回模型:首因峰、近因峰与随长度下沉的地板」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-255 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026255,
author = {Fan, Yafu},
title = {Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}