5D Library › 检索增强生成几何、倒数秩融合与多智能体误差级联 · English
第二章 词法与稠密检索:各自的盲区与倒数秩融合的平滑常数
两种检索,两种失败
稀疏的词法检索(以 BM25 为代表)按查询词与文本的精确重合打分 (Robertson and Zaragoza 2009);稠密检索按语义向量的相似度打分 (Karpukhin et al. 2020)。前者认得字面、认不得意思,后者认得意思、认不得字面。在含有错误代码、零件编号、版本号或罕见术语的查询上,稠密检索有一个结构性的弱点:分词器把这些字符串切成常见的子词,不同编号的向量几乎相同,内积无法分辨它们。本章先用一个合成语料把两种失败并排展示,再看把两者合起来的最常用方法——倒数秩融合(RRF)——在什么条件下有效,以及它的平滑常数 应当取多少。
盲区并排
推导
# Chapter 2. p04: Lexical (BM25) and dense retrieval fail on different
# queries.
# Synthetic corpus: 2,000 chunks in 50 topics. Each chunk has 20 topic
# words, 3 detail words of its own, and a unique error code such as
# E1043. Two query types, 200 each:
# paraphrase 3 topic synonyms + 3 paraphrased detail words - no word
# in common with the gold chunk
# exact code "what does code E1043 mean" - the code is the only clue
# BM25 (k1 = 1.2, b = 0.75) scores exact token overlap. The dense model
# (d = 64) maps a word and its synonym/paraphrase to nearby vectors, but
# like a subword tokenizer sees every code as almost the same token: a
# shared "code" vector plus only 0.15 of a code-specific part. A chunk
# or query vector is the normalised mean of its word vectors.
# Hybrid: reciprocal rank fusion, score = sum 1 / (60 + rank), done two
# ways: RRF-all fuses complete rankings (ties broken at random); RRF-100
# fuses each ranker's top-100 list only, and BM25 lists only chunks with
# a positive score - the usual implementation.
import math
import random
D, TOPICS, DOCS = 64, 50, 2000
def unit(rng):
v = [rng.gauss(0, 1) for _ in range(D)]
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
def mix(*parts):
v = [0.0] * D
for w, vec in parts:
v = [a + w * b for a, b in zip(v, vec)]
return v
def norm(v):
n = math.sqrt(sum(x * x for x in v)) or 1.0
return [x / n for x in v]
class Corpus:
def __init__(self, rng):
self.rng = rng
self.topic = [unit(rng) for _ in range(TOPICS)]
self.code_dir = unit(rng)
self.vec = {}
self.docs = []
for i in range(DOCS):
t = i % TOPICS
words = ["t{}w{}".format(t, rng.randrange(40)) for _ in range(20)]
words += ["d{}x{}".format(i, k) for k in range(3)]
words += ["E{}".format(1000 + i)]
self.docs.append((t, words))
self.df = {}
for _, ws in self.docs:
for w in set(ws):
self.df[w] = self.df.get(w, 0) + 1
self.avg = sum(len(ws) for _, ws in self.docs) / DOCS
self.emb = [self.embed(ws) for _, ws in self.docs]
def wvec(self, w):
if w in self.vec:
return self.vec[w]
r = self.rng
if w.startswith("t"): # topic word or synonym
t = int(w[1:].replace("s", "w").split("w")[0])
v = mix((1.0, self.topic[t]), (0.5, unit(r)))
elif w.startswith("d"): # detail word/paraphrase
base = w.replace("p", "x")
if base not in self.vec:
self.vec[base] = unit(r)
v = mix((1.0, self.vec[base]), (0.3, unit(r)))
if w == base:
return self.vec[base]
elif w.startswith("E"): # code: almost one token
v = mix((1.0, self.code_dir), (0.15, unit(r)))
else:
v = unit(r)
self.vec[w] = v
return v
def embed(self, words):
return norm(mix(*[(1.0, self.wvec(w)) for w in words]))
def bm25(self, q):
out = []
for i, (_, ws) in enumerate(self.docs):
s = 0.0
for w in set(q):
f = ws.count(w)
if f:
n = self.df[w]
idf = math.log(1 + (DOCS - n + 0.5) / (n + 0.5))
s += idf * f * 2.2 / (f + 1.2 * (0.25 + 0.75 * len(ws)
/ self.avg))
out.append(s)
return out
def dense(self, q):
qv = self.embed(q)
return [sum(a * b for a, b in zip(qv, e)) for e in self.emb]
def ranks(scores, rng):
order = sorted(range(len(scores)),
key=lambda i: (-scores[i], rng.random()))
r = [0] * len(scores)
for pos, i in enumerate(order):
r[i] = pos + 1
return r
if __name__ == "__main__":
rng = random.Random(61004)
c = Corpus(rng)
res = {}
for qtype in ("paraphrase", "exact code"):
hits = {"BM25": [0, 0], "dense": [0, 0], "RRF-all": [0, 0],
"RRF-100": [0, 0]}
for _ in range(200):
g = rng.randrange(DOCS)
t = c.docs[g][0]
if qtype == "paraphrase":
q = ["t{}s{}".format(t, rng.randrange(40)) for _ in range(3)]
q += ["d{}p{}".format(g, k) for k in range(3)]
else:
q = ["what", "does", "code", "E{}".format(1000 + g), "mean"]
sb = c.bm25(q)
rb = ranks(sb, rng)
rd = ranks(c.dense(q), rng)
fused = [1 / (60 + a) + 1 / (60 + b) for a, b in zip(rb, rd)]
rf = ranks(fused, rng)
f100 = [(1 / (60 + a) if a <= 100 and s > 0 else 0.0)
+ (1 / (60 + b) if b <= 100 else 0.0)
for a, b, s in zip(rb, rd, sb)]
r100 = ranks(f100, rng)
for name, r in (("BM25", rb), ("dense", rd), ("RRF-all", rf),
("RRF-100", r100)):
hits[name][0] += r[g] == 1
hits[name][1] += r[g] <= 10
res[qtype] = hits
print(" query type ranker recall@1 recall@10")
for qtype, hits in res.items():
for name, (h1, h10) in hits.items():
print(" {:<13} {:<7} {:7.3f} {:7.3f}".format(
qtype, name, h1 / 200, h10 / 200))
print("\nReading: BM25 cannot find a chunk that shares no words with")
print("the query; the dense model cannot tell one code from another,")
print("because every code is nearly the same vector. Fusion is not")
print("automatically the best of both: fusing full rankings lets the")
print("blind ranker's noise outvote the one that knows. Fusing short")
print("lists, with zero-score chunks left out, repairs paraphrase")
print("queries completely but leaves exact-code queries a near coin")
print("toss: the dense top hit ties with the true chunk.")
# query type ranker recall@1 recall@10
# paraphrase BM25 0.005 0.005
# paraphrase dense 0.980 1.000
# paraphrase RRF-all 0.055 0.740
# paraphrase RRF-100 0.980 1.000
# exact code BM25 1.000 1.000
# exact code dense 0.000 0.005
# exact code RRF-all 0.055 0.750
# exact code RRF-100 0.475 1.000
#
# Reading: BM25 cannot find a chunk that shares no words with
# the query; the dense model cannot tell one code from another,
# because every code is nearly the same vector. Fusion is not
# automatically the best of both: fusing full rankings lets the
# blind ranker's noise outvote the one that knows. Fusing short
# lists, with zero-score chunks left out, repairs paraphrase
# queries completely but leaves exact-code queries a near coin
# toss: the dense top hit ties with the true chunk.在改写式查询上(与正确块没有一个共同的词),BM25 的 recall@1 只有 0.005,稠密检索是 0.980。在精确代码查询上正好相反:BM25 是 1.000,稠密检索是 0.000——它把每个代码都看成几乎同一个向量,代码之间的差别只占向量的很小一部分,淹没在主题的差异里。这是对「纯向量内积在精确匹配上存在分辨率下界」的直接演示。
融合并不自动得到两者之长。把两份完整排名直接融合(RRF-all),两类查询的 recall@1 都只剩 0.055:对每一类查询,总有一个检索器在乱排,它的噪声与另一个检索器的正确信号获得同样的权重。按通常的做法只融合各自的前 100 名、并去掉 BM25 得分为零的块(RRF-100),改写式查询完全恢复(0.980),但精确代码查询只到 0.475:稠密检索排第一的那个错误块,与 BM25 排第一的正确块,在融合分数上打成平手,接近抛硬币。这个 0.475 来自等权重下的平局退化:正确块只靠 BM25 一路命中(单路第一名),稠密检索排第一的错误块也只靠一路命中,两者的 RRF 分数同为 ,谁排第一只取决于平局的打破方式。recall@10 都是 1.000,所以正确块总在前十名里——问题只在第一名。
注
这对实践意味着什么。 融合两个检索器之前,先按查询类型看各自的盲区。对已知含有标识符的查询(可以用简单的正则判断),给词法检索更高的权重或直接走词法通道,往往比对所有查询一视同仁的融合更好。
平滑常数 k
推导
倒数秩融合。 对每个块,融合分数为 是它在第 个检索器中的名次 (Cormack et al. 2009)。 小时,第一名与第二名的分数差别很大,融合由各检索器的头部决定; 大时,分数随名次变化平缓,融合接近于名次的平均。Cormack 等人在原文中使用 。
推导
# Chapter 2. p05: How the smoothing constant k in RRF moves head and tail.
# Two rankers score 1,000 chunks. Irrelevant chunks get standard-normal
# scores; the gold chunk gets mu plus noise. The two rankers' noise can
# be correlated (rho), as when both are fooled by the same chunk. Fused
# score = sum over rankers of 1 / (k + rank). Three scenarios, 2,000
# queries each:
# S1 equal mu = 2.5 and 2.5, rho = 0
# S2 unequal mu = 3.5 and 1.0, rho = 0 (one strong, one weak)
# S3 shared mu = 2.5 and 2.5, rho = 0.6
# For comparison, a convex blend of the raw scores, alpha * a + (1 -
# alpha) * b, with the best alpha on a 0.05 grid (it needs comparable
# scores, which real rankers do not give for free).
import math
import random
N, Q = 1000, 2000
KS = (1, 5, 20, 60, 200, 1000)
def scenario(rng, mu_a, mu_b, rho):
out = []
for _ in range(Q):
a, b = [], []
for _ in range(N):
z = rng.gauss(0, 1)
a.append(z)
b.append(rho * z + math.sqrt(1 - rho * rho) * rng.gauss(0, 1))
a[0] += mu_a
b[0] += mu_b
out.append((a, b))
return out
def rank_of_gold(score):
g = score[0]
return 1 + sum(1 for s in score[1:] if s > g)
def rank_list(score):
order = sorted(range(len(score)), key=lambda i: -score[i])
r = [0] * len(score)
for p, i in enumerate(order):
r[i] = p + 1
return r
def metrics(golds):
n = len(golds)
return (sum(r == 1 for r in golds) / n, sum(1 / r for r in golds) / n,
sum(r <= 20 for r in golds) / n)
if __name__ == "__main__":
rng = random.Random(61005)
for name, ma, mb, rho in (("S1 equal", 2.5, 2.5, 0.0),
("S2 unequal", 3.5, 1.0, 0.0),
("S3 shared", 2.5, 2.5, 0.6)):
data = scenario(rng, ma, mb, rho)
ranked = [(rank_list(a), rank_list(b), a, b) for a, b in data]
print("{} (alone: A r@1 {:.3f}, B r@1 {:.3f})".format(
name, metrics([ra[0] for ra, _, _, _ in ranked])[0],
metrics([rb[0] for _, rb, _, _ in ranked])[0]))
print(" k recall@1 MRR recall@20")
for k in KS:
g = [rank_of_gold([1 / (k + x) + 1 / (k + y)
for x, y in zip(ra, rb)])
for ra, rb, _, _ in ranked]
r1, mrr, r20 = metrics(g)
print(" {:5d} {:.3f} {:.3f} {:.3f}".format(
k, r1, mrr, r20))
best = max((metrics([rank_of_gold([al * x + (1 - al) * y
for x, y in zip(a, b)])
for _, _, a, b in ranked])[1], al)
for al in [i / 20 for i in range(21)])
print(" best score blend: alpha {:.2f}, MRR {:.3f}\n".format(
best[1], best[0]))
print("Reading: with two equal rankers fusion more than doubles")
print("recall@1, but k = 1 is clearly worse than k = 20 to 1000.")
print("With one strong and one weak ranker, every k does worse at the")
print("head than the strong ranker alone - small k loses least - and")
print("from k = 200 the tail collapses too. With shared errors fusion")
print("gains little whatever k is. In none of the three is k = 60 the")
print("best for recall@1; where scores are comparable, a tuned blend")
print("beats every k.")
# S1 equal (alone: A r@1 0.229, B r@1 0.247)
# k recall@1 MRR recall@20
# 1 0.425 0.516 0.847
# 5 0.507 0.588 0.855
# 20 0.586 0.679 0.881
# 60 0.560 0.667 0.917
# 200 0.547 0.650 0.911
# 1000 0.541 0.642 0.895
# best score blend: alpha 0.50, MRR 0.713
#
# S2 unequal (alone: A r@1 0.604, B r@1 0.017)
# k recall@1 MRR recall@20
# 1 0.478 0.601 0.892
# 5 0.470 0.601 0.897
# 20 0.297 0.486 0.907
# 60 0.211 0.353 0.903
# 200 0.197 0.297 0.640
# 1000 0.191 0.279 0.567
# best score blend: alpha 0.80, MRR 0.738
#
# S3 shared (alone: A r@1 0.243, B r@1 0.229)
# k recall@1 MRR recall@20
# 1 0.309 0.412 0.729
# 5 0.321 0.432 0.737
# 20 0.319 0.434 0.755
# 60 0.323 0.434 0.756
# 200 0.324 0.434 0.751
# 1000 0.324 0.433 0.748
# best score blend: alpha 0.55, MRR 0.443
#
# Reading: with two equal rankers fusion more than doubles
# recall@1, but k = 1 is clearly worse than k = 20 to 1000.
# With one strong and one weak ranker, every k does worse at the
# head than the strong ranker alone - small k loses least - and
# from k = 200 the tail collapses too. With shared errors fusion
# gains little whatever k is. In none of the three is k = 60 the
# best for recall@1; where scores are comparable, a tuned blend
# beats every k.三种情形给出三种不同的答案。两个检索器同样好(S1)时,融合让 recall@1 从单个检索器的 0.229 与 0.247 升到最高 0.586(); 明显更差(0.425), 在 20 到 1000 之间相差不大。一强一弱(S2)时,每一个 在头部都不如强检索器单独使用(0.604): 越小损失越少( 时 0.478), 时只有 0.211,从 起连 recall@20 也崩塌(0.640)。两个检索器犯同样的错(S3)时,融合几乎没有收益, 取多少都差不多。
在三种情形里, 都不是 recall@1 的最优值。若两个检索器的分数可以放到同一尺度上,调好权重的分数凸组合(最优 )在 S1 与 S2 中都胜过任何 (MRR 0.713 与 0.738)。所以对提纲的修正是:不存在对所有分布都最优的 ;它取决于检索器的相对强弱与错误的相关程度,需要在自己的查询上调。
假说 · 尚未验证
作者的假说(尚未验证): 在真实的混合检索系统中,按查询是否含有标识符分两类分别调 (或分别设定权重),比全局使用 在 recall@1 上至少高 3 个百分点。什么结果会证明它错:在预先登记的评测集上,分类调参与全局 的配对差异的 95% 置信区间上限低于 3 个百分点。
小结
词法检索认不出改写,稠密检索认不出编号;两者的盲区互补,但融合不会自动取长补短。融合完整排名时,盲的一方会用噪声抵消知道的一方;只融合短列表并去掉零分块,可以修复改写查询,却让编号查询停在抛硬币附近。RRF 的平滑常数 没有普遍最优值:两检索器相当时 从 20 起差别不大,一强一弱时任何 都不如强者单用,错误相关时融合几乎无益。
参考文献
Fan, Y. (2026). Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades: High-Dimensional Concentration of Measure, Hybrid Lexical-Dense Retrieval Bounds and Verifier-Gated Fixed Points [检索增强生成几何、倒数秩融合与多智能体误差级联] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090170 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).