5D Library › Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents · 中文
Chapter 4. The Context Budget: Lagrangian Water-Filling and the Best Number of Passages
How many passages to retrieve?
More passages are not always better. Language models make poorer use of information in the middle of long contexts (Liu et al. 2024), and extra passages distract. The program lets the needed passage appear in the top with probability (the smaller , the better the retriever); with the passage present the model answers correctly with probability , without it from its prior with probability 0.30; each passage also has a cost .
Calculation
# Chapter 4. p08: How many passages to retrieve: recall against distraction.
# The passage that answers a question is among the top k retrieved with
# probability recall(k) = 1 - exp(-k / kappa); kappa measures how good
# the retriever is (small = good). With the passage present the model
# answers correctly with probability 0.90 - 0.01 (k - 1) (each extra
# passage distracts a little); without it, from its prior, 0.30. Each
# 512-token passage also costs c in accuracy-equivalent units (latency,
# money): c = 0 or 0.002. The program finds the k from 1 to 40 that
# maximises accuracy - c k, for kappa = 2, 4 and 8.
import math
def acc(k, kappa):
rec = 1 - math.exp(-k / kappa)
return rec * (0.90 - 0.01 * (k - 1)) + (1 - rec) * 0.30
if __name__ == "__main__":
print(" kappa cost best k accuracy recall at best k")
for kappa in (2, 4, 8):
for c in (0.0, 0.002):
k = max(range(1, 41), key=lambda j: acc(j, kappa) - c * j)
print(" {:5d} {:6.3f} {:8d} {:10.4f} {:18.4f}".format(
kappa, c, k, acc(k, kappa), 1 - math.exp(-k / kappa)))
print("\nReading: more passages are not always better. Distraction")
print("caps the useful k at a handful for a good retriever and at")
print("about twice that for a weak one; a weak retriever cannot be")
print("rescued by a larger window, only by better ranking.")
# kappa cost best k accuracy recall at best k
# 2 0.000 7 0.8237 0.9698
# 2 0.002 6 0.8226 0.9502
# 4 0.000 10 0.7681 0.9179
# 4 0.002 10 0.7681 0.9179
# 8 0.000 15 0.6895 0.8466
# 8 0.002 14 0.6883 0.8262
#
# Reading: more passages are not always better. Distraction
# caps the useful k at a handful for a good retriever and at
# about twice that for a weak one; a weak retriever cannot be
# rescued by a larger window, only by better ranking.A good retriever () does best with only 7 passages, at accuracy 0.8237; a middling one () with 10, at 0.7681; a poor one () with 15, at only 0.6895. A poor retriever cannot be rescued by a larger window, only by better ranking.
Hypothesis · not established
The author’s interpretation (not yet tested): in the author’s system, limiting the profile to a summary of about 2,048 tokens and retrieving 6 to 10 passages is more accurate than putting the full profile and many passages into a long window. What would show this wrong: if, in the blind comparison, the long-window, full-profile version won clearly on user preference, the diminishing-returns assumption of this chapter would not hold for real models.
Summary
Under diminishing returns, the optimal division of the context is a water-filling solution: from 32k the profile stays at its 8 KB cap, and the rest goes to retrieval and history by weight. The best number of passages is set jointly by retrieval quality and distraction: a good retriever needs only 7, and a poor one cannot catch up however many it takes.
References
Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents: Multidimensional State Vectors, Personalized Prompt Conditioning and Cross-Corpus Synthesis [五维灵商量化架构、先验条件化生成与检索增强认知体] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).