关于「位置召回模型:首因峰、近因峰与随长度下沉的地板」,《中段遗忘与接力阈值》给出的定量结论是什么?

结论:程序 p31 用双指数模型分别拟合三十篇与二十篇数据,首因峰高度都是 0.249,衰减长度 2.5 与 2.2 篇,只有地板从 0.609 降到 0.578。程序 p32 把这一结构写成联合模型,五个参数拟合十二个格子,误差 0.0105,并外推出捕获曲线:四篇 0.908,三十二篇 0.612,两百五十六篇 0.477。这条曲线的每一个数都可以由经营者用自己的读者重测,第八章会给出方法。有了这条曲线,我们就能问接力的问题了:把长材料切成短段、写摘要、再往下传,每传一层会丢多少,传几层就不划算了?

数据口径:书中任务、材料与参数均为示意,模型常数来自 2023 年的一个模型;全部计算由书中程序生成,不调用任何模型接口;本书是非个人化经营建议,不评测、不推荐任何模型或服务。 书中以标注框区分已发表的证据、可运行的 Python 计算(21 段程序)、原典与作者的假说。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0105, 0.477, 0.578, 0.612, 0.908, 32

joint fit: tau 2.4M0 0.752K 0.0505A 0.251B 0.128
rmse over 12 cells: 0.0105 (relative recall)
nfloorrho(1)rho(mid)rho(n)capture c(n)
10.7521.0001.0001.0001.000
20.7171.0001.0001.0001.000
40.6830.9710.9040.8830.908
80.6480.9060.7440.7890.782
160.6130.8640.6310.7410.682
320.5780.8290.5780.7060.612
640.5430.7940.5430.6710.560
1280.5080.7590.5080.6360.516
2560.4730.7240.4730.6010.477

核心公式

\[\rho(i, n) = m + A\, e^{-(i-1)/\tau} + B\, e^{-(n-i)/\tau},\]
查看完整验算程序
# Chapter 3. p32: one model for both lengths, and the capture curve.
# Joint fit (ours) to the 20- and 30-document GPT-3.5-Turbo curves of
# Liu et al. (2024), relative to oracle accuracy:
#   rho(i, n) = M0 - K ln n + A exp(-(i-1)/tau) + B exp(-(n-i)/tau),
# capped at 1. The floor falls with log length (dilution); the two
# bumps keep a fixed size in documents. Grid on tau, least squares on
# M0, K, A, B. Capture c(n) = mean of rho over the n positions: the
# expected recall of a fact whose position is not known in advance.
# Extrapolation beyond 30 documents is illustrative only.
import math

DATA = (
    (30, (1, 5, 10, 15, 20, 25, 30),
     (73.4, 55.1, 50.5, 50.9, 51.8, 54.9, 63.7), 88.6),
    (20, (1, 5, 10, 15, 20), (75.8, 57.2, 53.8, 55.4, 63.2), 88.3),
)


def solve(a, b):
    k = len(b)
    m = [row[:] + [v] for row, v in zip(a, b)]
    for c in range(k):
        p = max(range(c, k), key=lambda r: abs(m[r][c]))
        m[c], m[p] = m[p], m[c]
        for r in range(k):
            if r != c:
                f = m[r][c] / m[c][c]
                m[r] = [x - f * y for x, y in zip(m[r], m[c])]
    return [m[i][k] / m[i][i] for i in range(k)]


def design(tau):
    rows, ys = [], []
    for n, xs, acc, orc in DATA:
        for x, v in zip(xs, acc):
            rows.append((1.0, -math.log(n), math.exp(-(x - 1) / tau),
                         math.exp(-(n - x) / tau)))
            ys.append(v / orc)
    return rows, ys


def fit(tau):
    rows, ys = design(tau)
    ata = [[math.fsum(r[i] * r[j] for r in rows) for j in range(4)]
           for i in range(4)]
    aty = [math.fsum(r[i] * y for r, y in zip(rows, ys))
           for i in range(4)]
    par = solve(ata, aty)
    res = [y - math.fsum(p * q for p, q in zip(par, r))
           for r, y in zip(rows, ys)]
    return math.sqrt(math.fsum(e * e for e in res) / len(res)), par


best = min(((fit(t / 10)[0], t / 10) for t in range(5, 151)))
rmse, par = fit(best[1])
m0, k, a, b = par
tau = best[1]
print("joint fit: tau %.1f  M0 %.3f  K %.4f  A %.3f  B %.3f"
      % (tau, m0, k, a, b))
print("rmse over 12 cells: %.4f (relative recall)" % rmse)


def rho(i, n):
    v = (m0 - k * math.log(n) + a * math.exp(-(i - 1) / tau)
         + b * math.exp(-(n - i) / tau))
    return min(1.0, v)


print("    n   floor  rho(1)  rho(mid)  rho(n)  capture c(n)")
for n in (1, 2, 4, 8, 16, 32, 64, 128, 256):
    c = math.fsum(rho(i, n) for i in range(1, n + 1)) / n
    print("%5d  %6.3f  %6.3f  %8.3f  %6.3f  %8.3f"
          % (n, m0 - k * math.log(n), rho(1, n), rho((n + 1) // 2, n),
             rho(n, n), c))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《长上下文注意力中段遗忘曲线、位置偏置与零损耗分段摘要接力阈值母典》第 3 章「位置召回模型:首因峰、近因峰与随长度下沉的地板」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-255 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026255,
  author    = {Fan, Yafu},
  title     = {Mid-Context Forgetting in Long-Context Attention: Positional Bias and the Threshold for Lossless Segment-Summary Relays},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论