关于「上下文预算:拉格朗日注水与最优检索段数」,《五维灵商、先验条件化与检索增强》给出的定量结论是什么?

结论:在边际递减的收益下,上下文的最优分配是一个注水解:档案在 32k 起就停在 8 KB 的上限,其余按权重分给检索与历史。最优检索段数由检索质量与注意力分散共同决定:好的检索器取 7 段即可,差的检索器取再多也追不上。

数据口径:书中以标注框区分证据、16 段可运行的 Python 程序、原典与假说。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 32

   window   budget  profile  retrieval  history    nu x 1e4  gain vs equal
     8192     6144     1508       3424     1212       5.531        0.2134
    32768    30720     2048      19448     9224       1.399        0.2321
   131072   129024     2048      84984    41992       0.345        0.2491
Reading: at 8k the profile already takes a quarter of the
budget; from 32k it sits at its 8 KB cap, and every further
token goes to retrieval and history in the ratio of their
weights. A larger window makes the profile's share shrink,
and the value of the last token (nu) falls sixteen-fold.
查看完整验算程序
# Chapter 4. p07: Splitting a context window: a Lagrangian water-filling.
# A context of T tokens, less 2,048 kept for the answer, is shared by
# the profile (P), retrieved passages (R) and conversation history (H).
# Each part has diminishing returns, U_i = a_i ln(1 + t_i / s_i), with
# weights a = (1.0, 3.0, 1.5) and half-saturation sizes s = (300,
# 2,000, 1,500) tokens; the profile cannot exceed its 8 KB file, about
# 2,048 tokens. Maximising sum U_i subject to sum t_i = B gives the
# Karush-Kuhn-Tucker solution t_i = clip(a_i / nu - s_i, 0, cap_i), with
# the multiplier nu (the value of one more token) found by bisection.
# Compared with an equal three-way split (the profile's surplus over
# its cap shared by the other two).
import math

A = (1.0, 3.0, 1.5)
S = (300.0, 2000.0, 1500.0)
CAP = (2048.0, math.inf, math.inf)
NAMES = ("profile", "retrieval", "history")

def alloc(nu):
    return [min(c, max(0.0, a / nu - s)) for a, s, c in zip(A, S, CAP)]

def solve(budget):
    lo, hi = 1e-9, 10.0
    for _ in range(200):
        mid = math.sqrt(lo * hi)
        if sum(alloc(mid)) > budget:
            lo = mid
        else:
            hi = mid
    return alloc(hi), hi

def utility(t):
    return sum(a * math.log(1 + x / s) for a, s, x in zip(A, S, t))

if __name__ == "__main__":
    print("   window   budget  profile  retrieval  history"
          "    nu x 1e4  gain vs equal")
    for T in (8192, 32768, 131072):
        b = T - 2048
        t, nu = solve(b)
        p0 = min(b / 3, CAP[0])
        eq = [p0, (b - p0) / 2, (b - p0) / 2]
        print("   {:6d} {:8d} {:8.0f} {:10.0f} {:8.0f} {:11.3f} {:13.4f}"
              .format(T, b, t[0], t[1], t[2], 1e4 * nu,
                      utility(t) - utility(eq)))
    print("\nReading: at 8k the profile already takes a quarter of the")
    print("budget; from 32k it sits at its 8 KB cap, and every further")
    print("token goes to retrieval and history in the ratio of their")
    print("weights. A larger window makes the profile's share shrink,")
    print("and the value of the last token (nu) falls sixteen-fold.")

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《五维灵商量化架构、先验条件化生成与检索增强认知体》第 4 章「上下文预算:拉格朗日注水与最优检索段数」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203

DOI: 10.5281/zenodo.24090203 · 阅读本章所在专著 · 5DL-B-076 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026076,
  author    = {Fan, Yafu},
  title     = {Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.24090203},
  url       = {https://doi.org/10.5281/zenodo.24090203}
}

本书其他章节结论