What does RAG and Multi-Agent Reliability conclude, in numbers, about “The Context Window: Position Decay, Evidence Dilution and the Best Number of Chunks”?
Answer: Chapter 3 uses a stylised reader model to show "lost in the middle" and evidence dilution: a gold chunk in the middle is answered correctly only about a third as often as at either end (0.098 against 0.288), and the best number of chunks falls from 8 to 2 as distractors grow plausible.
Scope of the numbers: All data are synthetic, generated by the book's programs; the numbers show mechanisms and orders of magnitude and are not measurements of any product.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.098, 0.288
| A. K | 20 chunks, s = 0.25, gold chunk placed at position p |
| position | 0: P(correct) 0.288 |
| position | 2: P(correct) 0.198 |
| position | 5: P(correct) 0.130 |
| position 10: P(correct) 0.098 | |
| position 15: P(correct) 0.146 | |
| position 19: P(correct) 0.288 | |
| B. retriever order, K chunks in the context | |
| K recall@K P(correct), s | 0.03 P(correct), s = 0.25 |
| 1 | 0.350 0.389 0.389 |
| 2 | 0.480 0.468 0.409 |
| 3 | 0.584 0.522 0.407 |
| 5 | 0.734 0.585 0.377 |
| 8 | 0.864 0.614 0.323 |
| 10 | 0.913 0.612 0.293 |
| 15 | 0.971 0.587 0.240 |
| 20 | 0.991 0.561 0.208 |
| 40 | 1.000 0.490 0.147 |
| s | 0.03: best K = 8 (P(correct) 0.614) |
| s | 0.25: best K = 2 (P(correct) 0.409) |
| Reading: the gold chunk is worth most at either end of the | |
| context and least in the middle. Adding chunks raises recall, | |
| but each distractor dilutes the gold chunk's share. With | |
| off-topic distractors accuracy keeps rising for a while; with | |
| plausible near-misses it peaks at very few chunks and then | |
| falls while recall is still climbing. The best K is a property | |
| of the corpus and the reader, not a constant. |
Key formula
Show the full program
# Chapter 3. p06: More context can mean less answer: position and dilution.
# A stylised reader model, not a measured one. The context holds K chunks
# in the retriever's order. Each position p (0 = first) gets attention
# weight w(p) = 0.25 + 0.75 max(exp(-p/3), exp(-(K-1-p)/3)) - high at both
# ends, low in the middle, the U-shape reported by Liu et al. (2024). The
# gold chunk pulls with strength 1, every distractor with strength s, and
# the model's own (possibly wrong) prior with 0.2. Probability of a
# correct answer when the gold chunk is present (Luce choice rule):
# w_gold / (w_gold + s sum w_distractors + 0.2)
# and 0.15 when it is absent. The retriever puts the gold chunk at rank
# 1 with probability 0.35, otherwise further down with a geometric tail:
# recall@K = 1 - 0.65 * 0.8^(K - 1). Two distractor strengths: s = 0.03
# (clearly off-topic chunks) and s = 0.25 (plausible near-misses).
import math
def w(p, k):
return 0.25 + 0.75 * max(math.exp(-p / 3), math.exp(-(k - 1 - p) / 3))
def correct(pos, k, s):
ws = [w(p, k) for p in range(k)]
g = ws[pos]
return g / (g + s * (sum(ws) - g) + 0.2)
def p_rank(r):
return 0.35 if r == 1 else 0.65 * 0.2 * 0.8 ** (r - 2)
def expected(k, s):
acc = sum(p_rank(r) * correct(r - 1, k, s) for r in range(1, k + 1))
miss = 1 - sum(p_rank(r) for r in range(1, k + 1))
return acc + miss * 0.15, 1 - miss
if __name__ == "__main__":
print("A. K = 20 chunks, s = 0.25, gold chunk placed at position p")
for p in (0, 2, 5, 10, 15, 19):
print(" position {:2d}: P(correct) {:.3f}".format(
p, correct(p, 20, 0.25)))
print("\nB. retriever order, K chunks in the context")
print(" K recall@K P(correct), s = 0.03 P(correct), s = 0.25")
best = {0.03: (0, 0), 0.25: (0, 0)}
for k in (1, 2, 3, 5, 8, 10, 15, 20, 40):
row = []
for s in (0.03, 0.25):
a, rec = expected(k, s)
best[s] = max(best[s], (a, k))
row.append(a)
print(" {:3d} {:6.3f} {:6.3f} {:6.3f}"
.format(k, rec, row[0], row[1]))
for s in (0.03, 0.25):
print(" s = {}: best K = {} (P(correct) {:.3f})".format(
s, best[s][1], best[s][0]))
print("\nReading: the gold chunk is worth most at either end of the")
print("context and least in the middle. Adding chunks raises recall,")
print("but each distractor dilutes the gold chunk's share. With")
print("off-topic distractors accuracy keeps rising for a while; with")
print("plausible near-misses it peaks at very few chunks and then")
print("falls while recall is still climbing. The best K is a property")
print("of the corpus and the reader, not a constant.")Single source and citation
Fan, Y. (2026). Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades (Version 1.0.0), Chapter 3: The Context Window: Position Decay, Evidence Dilution and the Best Number of Chunks. 5D Library. https://doi.org/10.5281/zenodo.24090170
DOI: 10.5281/zenodo.24090170 · Read the monograph · 5DL-B-061 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026061,
author = {Fan, Yafu},
title = {Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.24090170},
url = {https://doi.org/10.5281/zenodo.24090170}
}Other chapter answers in this book
- Chapter 1 · High-Dimensional Geometry: Concentration of Measure, Effective Dimension and Hub Chunks
- Chapter 2 · Lexical and Dense Retrieval: Their Blind Spots and the Smoothing Constant of Reciprocal Rank Fusion
- Chapter 4 · Multi-Agent Pipelines: Error Cascades and the Verifier-Gated Absorbing Markov Chain
- Chapter 5 · Verifier Independence and the Cost of Retries
- Chapter 6 · The Statistics of Evaluation Itself: Sample Size, Judge Bias and the Winner's Curse
- Chapter 7 · The Routing of the *Qilue* and Han Fei's *Canyan*: Two Ancient Methods of Retrieval and Verification
- Chapter 8 · A Five-Step Check and a Pre-Registered Offline Evaluation