5D Library › 五维灵商量化架构、先验条件化生成与检索增强认知体 · English
第四章 上下文预算:拉格朗日注水与最优检索段数
三方分一个窗口
上下文窗口是有限的。一次回答要在三样东西之间分配 token:用户档案(P)、检索到的段落(R)与对话历史(H)。每一样都有边际递减的收益:档案写到一定程度、检索到一定数量、历史回溯到一定长度之后,再多也没有多少用。程序把收益写成 ,权重 ,半饱和长度 个 token;档案不能超过它的 8 KB 文件,约 2,048 个 token;窗口中另留 2,048 个 token 给回答。
在总预算 的约束下最大化 ,拉格朗日条件给出「注水」解 ,其中乘子 是最后一个 token 的边际价值,用二分法求得 (Boyd and Vandenberghe 2004)。
推导
# Chapter 4. p07: Splitting a context window: a Lagrangian water-filling.
# A context of T tokens, less 2,048 kept for the answer, is shared by
# the profile (P), retrieved passages (R) and conversation history (H).
# Each part has diminishing returns, U_i = a_i ln(1 + t_i / s_i), with
# weights a = (1.0, 3.0, 1.5) and half-saturation sizes s = (300,
# 2,000, 1,500) tokens; the profile cannot exceed its 8 KB file, about
# 2,048 tokens. Maximising sum U_i subject to sum t_i = B gives the
# Karush-Kuhn-Tucker solution t_i = clip(a_i / nu - s_i, 0, cap_i), with
# the multiplier nu (the value of one more token) found by bisection.
# Compared with an equal three-way split (the profile's surplus over
# its cap shared by the other two).
import math
A = (1.0, 3.0, 1.5)
S = (300.0, 2000.0, 1500.0)
CAP = (2048.0, math.inf, math.inf)
NAMES = ("profile", "retrieval", "history")
def alloc(nu):
return [min(c, max(0.0, a / nu - s)) for a, s, c in zip(A, S, CAP)]
def solve(budget):
lo, hi = 1e-9, 10.0
for _ in range(200):
mid = math.sqrt(lo * hi)
if sum(alloc(mid)) > budget:
lo = mid
else:
hi = mid
return alloc(hi), hi
def utility(t):
return sum(a * math.log(1 + x / s) for a, s, x in zip(A, S, t))
if __name__ == "__main__":
print(" window budget profile retrieval history"
" nu x 1e4 gain vs equal")
for T in (8192, 32768, 131072):
b = T - 2048
t, nu = solve(b)
p0 = min(b / 3, CAP[0])
eq = [p0, (b - p0) / 2, (b - p0) / 2]
print(" {:6d} {:8d} {:8.0f} {:10.0f} {:8.0f} {:11.3f} {:13.4f}"
.format(T, b, t[0], t[1], t[2], 1e4 * nu,
utility(t) - utility(eq)))
print("\nReading: at 8k the profile already takes a quarter of the")
print("budget; from 32k it sits at its 8 KB cap, and every further")
print("token goes to retrieval and history in the ratio of their")
print("weights. A larger window makes the profile's share shrink,")
print("and the value of the last token (nu) falls sixteen-fold.")
# window budget profile retrieval history nu x 1e4 gain vs equal
# 8192 6144 1508 3424 1212 5.531 0.2134
# 32768 30720 2048 19448 9224 1.399 0.2321
# 131072 129024 2048 84984 41992 0.345 0.2491
#
# Reading: at 8k the profile already takes a quarter of the
# budget; from 32k it sits at its 8 KB cap, and every further
# token goes to retrieval and history in the ratio of their
# weights. A larger window makes the profile's share shrink,
# and the value of the last token (nu) falls sixteen-fold.8k 窗口(预算 6144)中,档案得到 1508 个 token,已占预算的四分之一;32k 起,档案停在它的上限 2,048,其余全部按权重分给检索与历史:128k 窗口中检索 84984、历史 41992。窗口越大,档案的份额越小,最后一个 token 的边际价值 从 5.531 降到 0.345(以 为单位),降了十六倍。与平均三分相比,最优分配的收益在各个窗口都高出 0.2134 到 0.2491 个单位。
换句话说,「更大的窗口让个人档案发挥更大作用」并不成立;更大的窗口主要让检索与历史受益。
检索几段最好
检索段落并不是越多越好。语言模型在长上下文中对中间位置的信息利用较差 (Liu et al. 2024),多放的段落会分散注意力。程序设所需的段落以 的概率出现在前 段中( 越小检索越好);段落在场时模型以 的概率答对,不在场时以 0.30 的概率靠先验答对;每段还有成本 。
推导
# Chapter 4. p08: How many passages to retrieve: recall against distraction.
# The passage that answers a question is among the top k retrieved with
# probability recall(k) = 1 - exp(-k / kappa); kappa measures how good
# the retriever is (small = good). With the passage present the model
# answers correctly with probability 0.90 - 0.01 (k - 1) (each extra
# passage distracts a little); without it, from its prior, 0.30. Each
# 512-token passage also costs c in accuracy-equivalent units (latency,
# money): c = 0 or 0.002. The program finds the k from 1 to 40 that
# maximises accuracy - c k, for kappa = 2, 4 and 8.
import math
def acc(k, kappa):
rec = 1 - math.exp(-k / kappa)
return rec * (0.90 - 0.01 * (k - 1)) + (1 - rec) * 0.30
if __name__ == "__main__":
print(" kappa cost best k accuracy recall at best k")
for kappa in (2, 4, 8):
for c in (0.0, 0.002):
k = max(range(1, 41), key=lambda j: acc(j, kappa) - c * j)
print(" {:5d} {:6.3f} {:8d} {:10.4f} {:18.4f}".format(
kappa, c, k, acc(k, kappa), 1 - math.exp(-k / kappa)))
print("\nReading: more passages are not always better. Distraction")
print("caps the useful k at a handful for a good retriever and at")
print("about twice that for a weak one; a weak retriever cannot be")
print("rescued by a larger window, only by better ranking.")
# kappa cost best k accuracy recall at best k
# 2 0.000 7 0.8237 0.9698
# 2 0.002 6 0.8226 0.9502
# 4 0.000 10 0.7681 0.9179
# 4 0.002 10 0.7681 0.9179
# 8 0.000 15 0.6895 0.8466
# 8 0.002 14 0.6883 0.8262
#
# Reading: more passages are not always better. Distraction
# caps the useful k at a handful for a good retriever and at
# about twice that for a weak one; a weak retriever cannot be
# rescued by a larger window, only by better ranking.好的检索器()最优只取 7 段,准确率 0.8237;一般的()取 10 段,0.7681;差的()取 15 段,也只有 0.6895。差的检索器不能靠更大的窗口挽救,只能靠更好的排序。
假说 · 尚未验证
作者的诠释(尚未验证): 在作者的系统中,把档案限制在约 2,048 个 token 的摘要、检索取 6 到 10 段,比把完整档案与大量段落都塞进长窗口更准确。什么会证明这一判断错:若在盲比实验中,长窗口、全量档案的版本在用户偏好上显著胜出,说明本章的边际递减假设对真实模型不成立。
小结
在边际递减的收益下,上下文的最优分配是一个注水解:档案在 32k 起就停在 8 KB 的上限,其余按权重分给检索与历史。最优检索段数由检索质量与注意力分散共同决定:好的检索器取 7 段即可,差的检索器取再多也追不上。
参考文献
Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents: Multidimensional State Vectors, Personalized Prompt Conditioning and Cross-Corpus Synthesis [五维灵商量化架构、先验条件化生成与检索增强认知体] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).