5D Library › 检索增强生成几何、倒数秩融合与多智能体误差级联 · English

第三章 上下文窗口:位置衰减、证据稀释与最佳块数

窗口越大,答得越好吗

长上下文模型让人倾向于把检索到的块尽量多地塞进窗口:多放几块,正确的那块被包含进来的机会就大。Liu 等人的实验显示了另一面:模型对上下文开头与结尾的信息利用得最好,对中间的信息利用得最差,呈 U 形;相关信息被埋在中间时,准确率可以低于不给任何文档 (Liu et al. 2024)。本章用一个程式化的读者模型,把「召回率上升」与「证据被稀释」这两股相反的力量放到一起,看准确率怎样随块数变化。

推导

程式化读者模型。 位置 pp 的注意权重为 w(p)=0.25+0.75max⁡(e−p/3,e−(K−1−p)/3)w(p) = 0.25 + 0.75\max(e^{-p/3}, e^{-(K-1-p)/3}),两端高、中间低。正确块的「拉力」为 1,每个干扰块为 ss,模型自身(可能错误的)先验为 0.2。正确块在场时,回答正确的概率按 Luce 选择规则 (Luce 1959) 为 wgoldwgold+s∑wdistractor+0.2,\frac{w_{\text{gold}}}{w_{\text{gold}} + s\sum w_{\text{distractor}} + 0.2} , 不在场时为 0.15。检索器把正确块排在第一的概率为 0.35,其余按几何分布落在后面。这是一个用来说明机制的模型,不是对任何具体语言模型的测量。

推导

# Chapter 3. p06: More context can mean less answer: position and dilution.
# A stylised reader model, not a measured one. The context holds K chunks
# in the retriever's order. Each position p (0 = first) gets attention
# weight w(p) = 0.25 + 0.75 max(exp(-p/3), exp(-(K-1-p)/3)) - high at both
# ends, low in the middle, the U-shape reported by Liu et al. (2024). The
# gold chunk pulls with strength 1, every distractor with strength s, and
# the model's own (possibly wrong) prior with 0.2. Probability of a
# correct answer when the gold chunk is present (Luce choice rule):
#     w_gold / (w_gold + s sum w_distractors + 0.2)
# and 0.15 when it is absent. The retriever puts the gold chunk at rank
# 1 with probability 0.35, otherwise further down with a geometric tail:
# recall@K = 1 - 0.65 * 0.8^(K - 1). Two distractor strengths: s = 0.03
# (clearly off-topic chunks) and s = 0.25 (plausible near-misses).
import math

def w(p, k):
    return 0.25 + 0.75 * max(math.exp(-p / 3), math.exp(-(k - 1 - p) / 3))

def correct(pos, k, s):
    ws = [w(p, k) for p in range(k)]
    g = ws[pos]
    return g / (g + s * (sum(ws) - g) + 0.2)

def p_rank(r):
    return 0.35 if r == 1 else 0.65 * 0.2 * 0.8 ** (r - 2)

def expected(k, s):
    acc = sum(p_rank(r) * correct(r - 1, k, s) for r in range(1, k + 1))
    miss = 1 - sum(p_rank(r) for r in range(1, k + 1))
    return acc + miss * 0.15, 1 - miss

if __name__ == "__main__":
    print("A. K = 20 chunks, s = 0.25, gold chunk placed at position p")
    for p in (0, 2, 5, 10, 15, 19):
        print("   position {:2d}: P(correct) {:.3f}".format(
            p, correct(p, 20, 0.25)))
    print("\nB. retriever order, K chunks in the context")
    print("     K   recall@K   P(correct), s = 0.03   P(correct), s = 0.25")
    best = {0.03: (0, 0), 0.25: (0, 0)}
    for k in (1, 2, 3, 5, 8, 10, 15, 20, 40):
        row = []
        for s in (0.03, 0.25):
            a, rec = expected(k, s)
            best[s] = max(best[s], (a, k))
            row.append(a)
        print("   {:3d}    {:6.3f}          {:6.3f}                {:6.3f}"
              .format(k, rec, row[0], row[1]))
    for s in (0.03, 0.25):
        print("   s = {}: best K = {} (P(correct) {:.3f})".format(
            s, best[s][1], best[s][0]))
    print("\nReading: the gold chunk is worth most at either end of the")
    print("context and least in the middle. Adding chunks raises recall,")
    print("but each distractor dilutes the gold chunk's share. With")
    print("off-topic distractors accuracy keeps rising for a while; with")
    print("plausible near-misses it peaks at very few chunks and then")
    print("falls while recall is still climbing. The best K is a property")
    print("of the corpus and the reader, not a constant.")
# A. K = 20 chunks, s = 0.25, gold chunk placed at position p
#    position  0: P(correct) 0.288
#    position  2: P(correct) 0.198
#    position  5: P(correct) 0.130
#    position 10: P(correct) 0.098
#    position 15: P(correct) 0.146
#    position 19: P(correct) 0.288
#
# B. retriever order, K chunks in the context
#      K   recall@K   P(correct), s = 0.03   P(correct), s = 0.25
#      1     0.350           0.389                 0.389
#      2     0.480           0.468                 0.409
#      3     0.584           0.522                 0.407
#      5     0.734           0.585                 0.377
#      8     0.864           0.614                 0.323
#     10     0.913           0.612                 0.293
#     15     0.971           0.587                 0.240
#     20     0.991           0.561                 0.208
#     40     1.000           0.490                 0.147
#    s = 0.03: best K = 8 (P(correct) 0.614)
#    s = 0.25: best K = 2 (P(correct) 0.409)
#
# Reading: the gold chunk is worth most at either end of the
# context and least in the middle. Adding chunks raises recall,
# but each distractor dilutes the gold chunk's share. With
# off-topic distractors accuracy keeps rising for a while; with
# plausible near-misses it peaks at very few chunks and then
# falls while recall is still climbing. The best K is a property
# of the corpus and the reader, not a constant.

A 段把正确块放在 20 块上下文的不同位置:在开头或结尾时回答正确的概率是 0.288,在正中间(位置 10)只有 0.098——U 形。B 段让上下文按检索顺序放入 KK 块。召回率随 KK 单调上升:K=1K = 1 时 0.350,K=10K = 10 时 0.913,K=40K = 40 时 1.000。但准确率不是:干扰块明显离题(s=0.03s = 0.03)时,准确率先升后降,在 K=8K = 8 时最高(0.614),到 K=40K = 40 降到 0.490;干扰块是似是而非的「近似命中」(s=0.25s = 0.25)时,最佳 KK 只有 2(0.409),K=20K = 20 时已降到 0.208,此时召回率却是 0.991。

所以「上下文塞入过多低相关块会稀释正确证据」是成立的,而最佳块数不是一个常数:它取决于干扰块与正确块有多像(语料的性质)以及读者对位置有多敏感(模型的性质)。

两种干扰强度下,回答正确的概率(实线)与召回率(虚线)随上下文块数 K 的变化;召回率一路上升,准确率先升后降。

把最好的块放在两端

推导

# Chapter 3. p07: Where to put the best chunks.
# Reader model and retriever of p06 (U-shaped position weights, gold
# chunk at rank 1 with probability 0.35, geometric tail). With K chunks
# fixed, three ways to lay out the retriever's ranks in the context:
#   forward   rank 1 first, rank K last
#   reverse   rank K first, rank 1 last (nearest the question)
#   edges     ranks 1, 3, 5, ... from the front and 2, 4, 6, ... from the
#             back, so the strongest chunks sit at both ends
import math

def w(p, k):
    return 0.25 + 0.75 * max(math.exp(-p / 3), math.exp(-(k - 1 - p) / 3))

def correct(pos, k, s):
    ws = [w(p, k) for p in range(k)]
    g = ws[pos]
    return g / (g + s * (sum(ws) - g) + 0.2)

def p_rank(r):
    return 0.35 if r == 1 else 0.65 * 0.2 * 0.8 ** (r - 2)

def layout(name, k):
    if name == "forward":
        return list(range(k))
    if name == "reverse":
        return list(range(k - 1, -1, -1))
    pos, front, back = [0] * k, 0, k - 1
    for r in range(k):
        if r % 2 == 0:
            pos[r], front = front, front + 1
        else:
            pos[r], back = back, back - 1
    return pos

def expected(k, s, name):
    pos = layout(name, k)
    acc = sum(p_rank(r + 1) * correct(pos[r], k, s) for r in range(k))
    miss = 1 - sum(p_rank(r) for r in range(1, k + 1))
    return acc + miss * 0.15

if __name__ == "__main__":
    print("   K    s      forward   reverse   edges    edges - forward")
    for k in (5, 10, 20):
        for s in (0.03, 0.25):
            f, r, e = (expected(k, s, n) for n in
                       ("forward", "reverse", "edges"))
            print("  {:3d}  {:4.2f}    {:.3f}     {:.3f}     {:.3f}"
                  "     {:+.3f}".format(k, s, f, r, e, e - f))
    print("\nReading: because the reader favours both ends, putting the")
    print("strongest chunks at both ends recovers part of what the middle")
    print("loses. The gain grows with K, since a longer context has more")
    print("middle; it cannot exceed what the retriever put in the list.")
    print("Forward and reverse tie only because this model's U is")
    print("symmetric; a real reader that favours the end nearest the")
    print("question would separate them.")
#    K    s      forward   reverse   edges    edges - forward
#     5  0.03    0.585     0.585     0.590     +0.005
#     5  0.25    0.377     0.377     0.384     +0.007
#    10  0.03    0.612     0.612     0.634     +0.022
#    10  0.25    0.293     0.293     0.313     +0.020
#    20  0.03    0.561     0.561     0.607     +0.047
#    20  0.25    0.208     0.208     0.237     +0.029
#
# Reading: because the reader favours both ends, putting the
# strongest chunks at both ends recovers part of what the middle
# loses. The gain grows with K, since a longer context has more
# middle; it cannot exceed what the retriever put in the list.
# Forward and reverse tie only because this model's U is
# symmetric; a real reader that favours the end nearest the
# question would separate them.

既然读者偏爱两端,就把检索分数最高的块交替放在开头与结尾(「两端」排法),把较弱的块放在中间。与按分数从高到低顺排相比,收益随 KK 增大:K=5K = 5 时只多 0.005 到 0.007,K=20K = 20、干扰离题时多 0.047。顺排与倒排在这个模型里完全相同,只是因为模型的 U 形是对称的;真实的模型往往更偏重靠近问题的一端,那时两者会分开。

注

重排只能分配,不能创造。 重排改变的是正确块落在哪个位置,不改变它是否被检索到。召回不到的块,放在哪里都没用;在决定 KK 与排法之前,先看检索器把正确块排在第几名。

假说 · 尚未验证

作者的假说(尚未验证): 在给定的语言模型与语料上,按「两端」排法放入检索结果,比按分数顺排的问答准确率提高的幅度,与上下文块数 KK 正相关,且在 K≤5K \le 5 时小于 1 个百分点。什么结果会证明它错:在预先登记的问答集上,K=20K = 20 时的提高幅度不大于 K=5K = 5 时,或 K≤5K \le 5 时的提高超过 1 个百分点。

小结

读者对上下文两端的利用优于中间。增加块数提高召回,但每个干扰块都稀释正确块的份额:干扰离题时准确率在几块之后达到峰值,干扰似是而非时在两块左右就达到峰值,此后召回仍在上升、准确率却在下降。把最强的块放在两端可以收回一部分中间的损失,块数越多收益越大;但重排不能弥补检索不到的块。

参考文献

Liu, Nelson F., Kevin Lin, John Hewitt, et al. 2024. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics 12: 157–73.
Luce, R. Duncan. 1959. Individual Choice Behavior: A Theoretical Analysis. Wiley.

Fan, Y. (2026). Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades: High-Dimensional Concentration of Measure, Hybrid Lexical-Dense Retrieval Bounds and Verifier-Gated Fixed Points [检索增强生成几何、倒数秩融合与多智能体误差级联] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090170 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).