What does Spiritual Quotient, Prior Conditioning and RAG Agents conclude, in numbers, about “The Context Budget: Lagrangian Water-Filling and the Best Number of Passages”?

Answer: Under diminishing returns, the optimal division of the context is a water-filling solution: from 32k the profile stays at its 8 KB cap, and the rest goes to retrieval and history by weight. The best number of passages is set jointly by retrieval quality and distraction: a good retriever needs only 7, and a poor one cannot catch up however many it takes.

Scope of the numbers: Marked boxes separate evidence, 16 runnable Python programs, classical sources and hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 32

   window   budget  profile  retrieval  history    nu x 1e4  gain vs equal
     8192     6144     1508       3424     1212       5.531        0.2134
    32768    30720     2048      19448     9224       1.399        0.2321
   131072   129024     2048      84984    41992       0.345        0.2491
Reading: at 8k the profile already takes a quarter of the
budget; from 32k it sits at its 8 KB cap, and every further
token goes to retrieval and history in the ratio of their
weights. A larger window makes the profile's share shrink,
and the value of the last token (nu) falls sixteen-fold.
Show the full program
# Chapter 4. p07: Splitting a context window: a Lagrangian water-filling.
# A context of T tokens, less 2,048 kept for the answer, is shared by
# the profile (P), retrieved passages (R) and conversation history (H).
# Each part has diminishing returns, U_i = a_i ln(1 + t_i / s_i), with
# weights a = (1.0, 3.0, 1.5) and half-saturation sizes s = (300,
# 2,000, 1,500) tokens; the profile cannot exceed its 8 KB file, about
# 2,048 tokens. Maximising sum U_i subject to sum t_i = B gives the
# Karush-Kuhn-Tucker solution t_i = clip(a_i / nu - s_i, 0, cap_i), with
# the multiplier nu (the value of one more token) found by bisection.
# Compared with an equal three-way split (the profile's surplus over
# its cap shared by the other two).
import math

A = (1.0, 3.0, 1.5)
S = (300.0, 2000.0, 1500.0)
CAP = (2048.0, math.inf, math.inf)
NAMES = ("profile", "retrieval", "history")

def alloc(nu):
    return [min(c, max(0.0, a / nu - s)) for a, s, c in zip(A, S, CAP)]

def solve(budget):
    lo, hi = 1e-9, 10.0
    for _ in range(200):
        mid = math.sqrt(lo * hi)
        if sum(alloc(mid)) > budget:
            lo = mid
        else:
            hi = mid
    return alloc(hi), hi

def utility(t):
    return sum(a * math.log(1 + x / s) for a, s, x in zip(A, S, t))

if __name__ == "__main__":
    print("   window   budget  profile  retrieval  history"
          "    nu x 1e4  gain vs equal")
    for T in (8192, 32768, 131072):
        b = T - 2048
        t, nu = solve(b)
        p0 = min(b / 3, CAP[0])
        eq = [p0, (b - p0) / 2, (b - p0) / 2]
        print("   {:6d} {:8d} {:8.0f} {:10.0f} {:8.0f} {:11.3f} {:13.4f}"
              .format(T, b, t[0], t[1], t[2], 1e4 * nu,
                      utility(t) - utility(eq)))
    print("\nReading: at 8k the profile already takes a quarter of the")
    print("budget; from 32k it sits at its 8 KB cap, and every further")
    print("token goes to retrieval and history in the ratio of their")
    print("weights. A larger window makes the profile's share shrink,")
    print("and the value of the last token (nu) falls sixteen-fold.")

Single source and citation

Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents (Version 1.0.0), Chapter 4: The Context Budget: Lagrangian Water-Filling and the Best Number of Passages. 5D Library. https://doi.org/10.5281/zenodo.24090203

DOI: 10.5281/zenodo.24090203 · Read the monograph · 5DL-B-076 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026076,
  author    = {Fan, Yafu},
  title     = {Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.24090203},
  url       = {https://doi.org/10.5281/zenodo.24090203}
}

Other chapter answers in this book