关于「上下文预算:拉格朗日注水与最优检索段数」,《五维灵商、先验条件化与检索增强》给出的定量结论是什么?
结论:在边际递减的收益下,上下文的最优分配是一个注水解:档案在 32k 起就停在 8 KB 的上限,其余按权重分给检索与历史。最优检索段数由检索质量与注意力分散共同决定:好的检索器取 7 段即可,差的检索器取再多也追不上。
数据口径:书中以标注框区分证据、16 段可运行的 Python 程序、原典与假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 32
window budget profile retrieval history nu x 1e4 gain vs equal
8192 6144 1508 3424 1212 5.531 0.2134
32768 30720 2048 19448 9224 1.399 0.2321
131072 129024 2048 84984 41992 0.345 0.2491
Reading: at 8k the profile already takes a quarter of the
budget; from 32k it sits at its 8 KB cap, and every further
token goes to retrieval and history in the ratio of their
weights. A larger window makes the profile's share shrink,
and the value of the last token (nu) falls sixteen-fold.查看完整验算程序
# Chapter 4. p07: Splitting a context window: a Lagrangian water-filling.
# A context of T tokens, less 2,048 kept for the answer, is shared by
# the profile (P), retrieved passages (R) and conversation history (H).
# Each part has diminishing returns, U_i = a_i ln(1 + t_i / s_i), with
# weights a = (1.0, 3.0, 1.5) and half-saturation sizes s = (300,
# 2,000, 1,500) tokens; the profile cannot exceed its 8 KB file, about
# 2,048 tokens. Maximising sum U_i subject to sum t_i = B gives the
# Karush-Kuhn-Tucker solution t_i = clip(a_i / nu - s_i, 0, cap_i), with
# the multiplier nu (the value of one more token) found by bisection.
# Compared with an equal three-way split (the profile's surplus over
# its cap shared by the other two).
import math
A = (1.0, 3.0, 1.5)
S = (300.0, 2000.0, 1500.0)
CAP = (2048.0, math.inf, math.inf)
NAMES = ("profile", "retrieval", "history")
def alloc(nu):
return [min(c, max(0.0, a / nu - s)) for a, s, c in zip(A, S, CAP)]
def solve(budget):
lo, hi = 1e-9, 10.0
for _ in range(200):
mid = math.sqrt(lo * hi)
if sum(alloc(mid)) > budget:
lo = mid
else:
hi = mid
return alloc(hi), hi
def utility(t):
return sum(a * math.log(1 + x / s) for a, s, x in zip(A, S, t))
if __name__ == "__main__":
print(" window budget profile retrieval history"
" nu x 1e4 gain vs equal")
for T in (8192, 32768, 131072):
b = T - 2048
t, nu = solve(b)
p0 = min(b / 3, CAP[0])
eq = [p0, (b - p0) / 2, (b - p0) / 2]
print(" {:6d} {:8d} {:8.0f} {:10.0f} {:8.0f} {:11.3f} {:13.4f}"
.format(T, b, t[0], t[1], t[2], 1e4 * nu,
utility(t) - utility(eq)))
print("\nReading: at 8k the profile already takes a quarter of the")
print("budget; from 32k it sits at its 8 KB cap, and every further")
print("token goes to retrieval and history in the ratio of their")
print("weights. A larger window makes the profile's share shrink,")
print("and the value of the last token (nu) falls sixteen-fold.")唯一数据源与引用
范雅福(Fan, Y.). (2026). 《五维灵商量化架构、先验条件化生成与检索增强认知体》第 4 章「上下文预算:拉格朗日注水与最优检索段数」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203
DOI: 10.5281/zenodo.24090203 · 阅读本章所在专著 · 5DL-B-076 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026076,
author = {Fan, Yafu},
title = {Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.24090203},
url = {https://doi.org/10.5281/zenodo.24090203}
}