What does RAG and Multi-Agent Reliability conclude, in numbers, about “The Context Window: Position Decay, Evidence Dilution and the Best Number of Chunks”?

Answer: Chapter 3 uses a stylised reader model to show "lost in the middle" and evidence dilution: a gold chunk in the middle is answered correctly only about a third as often as at either end (0.098 against 0.288), and the best number of chunks falls from 8 to 2 as distractors grow plausible.

Scope of the numbers: All data are synthetic, generated by the book's programs; the numbers show mechanisms and orders of magnitude and are not measurements of any product.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.098, 0.288

A. K20 chunks, s = 0.25, gold chunk placed at position p
position0: P(correct) 0.288
position2: P(correct) 0.198
position5: P(correct) 0.130
position 10: P(correct) 0.098
position 15: P(correct) 0.146
position 19: P(correct) 0.288
B. retriever order, K chunks in the context
K recall@K P(correct), s0.03 P(correct), s = 0.25
10.350 0.389 0.389
20.480 0.468 0.409
30.584 0.522 0.407
50.734 0.585 0.377
80.864 0.614 0.323
100.913 0.612 0.293
150.971 0.587 0.240
200.991 0.561 0.208
401.000 0.490 0.147
s0.03: best K = 8 (P(correct) 0.614)
s0.25: best K = 2 (P(correct) 0.409)
Reading: the gold chunk is worth most at either end of the
context and least in the middle. Adding chunks raises recall,
but each distractor dilutes the gold chunk's share. With
off-topic distractors accuracy keeps rising for a while; with
plausible near-misses it peaks at very few chunks and then
falls while recall is still climbing. The best K is a property
of the corpus and the reader, not a constant.

Key formula

\[\frac{w_{\text{gold}}}{w_{\text{gold}} + s\sum w_{\text{distractor}} + 0.2} ,\]
Show the full program
# Chapter 3. p06: More context can mean less answer: position and dilution.
# A stylised reader model, not a measured one. The context holds K chunks
# in the retriever's order. Each position p (0 = first) gets attention
# weight w(p) = 0.25 + 0.75 max(exp(-p/3), exp(-(K-1-p)/3)) - high at both
# ends, low in the middle, the U-shape reported by Liu et al. (2024). The
# gold chunk pulls with strength 1, every distractor with strength s, and
# the model's own (possibly wrong) prior with 0.2. Probability of a
# correct answer when the gold chunk is present (Luce choice rule):
#     w_gold / (w_gold + s sum w_distractors + 0.2)
# and 0.15 when it is absent. The retriever puts the gold chunk at rank
# 1 with probability 0.35, otherwise further down with a geometric tail:
# recall@K = 1 - 0.65 * 0.8^(K - 1). Two distractor strengths: s = 0.03
# (clearly off-topic chunks) and s = 0.25 (plausible near-misses).
import math

def w(p, k):
    return 0.25 + 0.75 * max(math.exp(-p / 3), math.exp(-(k - 1 - p) / 3))

def correct(pos, k, s):
    ws = [w(p, k) for p in range(k)]
    g = ws[pos]
    return g / (g + s * (sum(ws) - g) + 0.2)

def p_rank(r):
    return 0.35 if r == 1 else 0.65 * 0.2 * 0.8 ** (r - 2)

def expected(k, s):
    acc = sum(p_rank(r) * correct(r - 1, k, s) for r in range(1, k + 1))
    miss = 1 - sum(p_rank(r) for r in range(1, k + 1))
    return acc + miss * 0.15, 1 - miss

if __name__ == "__main__":
    print("A. K = 20 chunks, s = 0.25, gold chunk placed at position p")
    for p in (0, 2, 5, 10, 15, 19):
        print("   position {:2d}: P(correct) {:.3f}".format(
            p, correct(p, 20, 0.25)))
    print("\nB. retriever order, K chunks in the context")
    print("     K   recall@K   P(correct), s = 0.03   P(correct), s = 0.25")
    best = {0.03: (0, 0), 0.25: (0, 0)}
    for k in (1, 2, 3, 5, 8, 10, 15, 20, 40):
        row = []
        for s in (0.03, 0.25):
            a, rec = expected(k, s)
            best[s] = max(best[s], (a, k))
            row.append(a)
        print("   {:3d}    {:6.3f}          {:6.3f}                {:6.3f}"
              .format(k, rec, row[0], row[1]))
    for s in (0.03, 0.25):
        print("   s = {}: best K = {} (P(correct) {:.3f})".format(
            s, best[s][1], best[s][0]))
    print("\nReading: the gold chunk is worth most at either end of the")
    print("context and least in the middle. Adding chunks raises recall,")
    print("but each distractor dilutes the gold chunk's share. With")
    print("off-topic distractors accuracy keeps rising for a while; with")
    print("plausible near-misses it peaks at very few chunks and then")
    print("falls while recall is still climbing. The best K is a property")
    print("of the corpus and the reader, not a constant.")

Single source and citation

Fan, Y. (2026). Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades (Version 1.0.0), Chapter 3: The Context Window: Position Decay, Evidence Dilution and the Best Number of Chunks. 5D Library. https://doi.org/10.5281/zenodo.24090170

DOI: 10.5281/zenodo.24090170 · Read the monograph · 5DL-B-061 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026061,
  author    = {Fan, Yafu},
  title     = {Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.24090170},
  url       = {https://doi.org/10.5281/zenodo.24090170}
}

Other chapter answers in this book