关于「词法与稠密检索:各自的盲区与倒数秩融合的平滑常数」,《检索增强与多智能体的可靠性》给出的定量结论是什么?
结论:第二章说明稠密检索认不出编号、词法检索认不出改写,融合完整排名时两类查询的 recall@1 都只剩 0.055,只融合短列表时编号查询停在 0.475;倒数秩融合的平滑常数 k = 60 在三种情形下都不是 recall@1 的最优值,一强一弱时任何 k 都不如强检索器单用(0.604)。
数据口径:全部数据为程序生成的合成数据,数字说明机制与量级,不是对任何产品的测量。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.055, 0.475
query type ranker recall@1 recall@10 paraphrase BM25 0.005 0.005 paraphrase dense 0.980 1.000 paraphrase RRF-all 0.055 0.740 paraphrase RRF-100 0.980 1.000 exact code BM25 1.000 1.000 exact code dense 0.000 0.005 exact code RRF-all 0.055 0.750 exact code RRF-100 0.475 1.000 Reading: BM25 cannot find a chunk that shares no words with the query; the dense model cannot tell one code from another, because every code is nearly the same vector. Fusion is not automatically the best of both: fusing full rankings lets the blind ranker's noise outvote the one that knows. Fusing short lists, with zero-score chunks left out, repairs paraphrase queries completely but leaves exact-code queries a near coin toss: the dense top hit ties with the true chunk.
核心公式
\[\mathrm{RRF}(d) = \sum_{i} \frac{1}{k + r_i(d)} ,\]
查看完整验算程序
# Chapter 2. p04: Lexical (BM25) and dense retrieval fail on different
# queries.
# Synthetic corpus: 2,000 chunks in 50 topics. Each chunk has 20 topic
# words, 3 detail words of its own, and a unique error code such as
# E1043. Two query types, 200 each:
# paraphrase 3 topic synonyms + 3 paraphrased detail words - no word
# in common with the gold chunk
# exact code "what does code E1043 mean" - the code is the only clue
# BM25 (k1 = 1.2, b = 0.75) scores exact token overlap. The dense model
# (d = 64) maps a word and its synonym/paraphrase to nearby vectors, but
# like a subword tokenizer sees every code as almost the same token: a
# shared "code" vector plus only 0.15 of a code-specific part. A chunk
# or query vector is the normalised mean of its word vectors.
# Hybrid: reciprocal rank fusion, score = sum 1 / (60 + rank), done two
# ways: RRF-all fuses complete rankings (ties broken at random); RRF-100
# fuses each ranker's top-100 list only, and BM25 lists only chunks with
# a positive score - the usual implementation.
import math
import random
D, TOPICS, DOCS = 64, 50, 2000
def unit(rng):
v = [rng.gauss(0, 1) for _ in range(D)]
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
def mix(*parts):
v = [0.0] * D
for w, vec in parts:
v = [a + w * b for a, b in zip(v, vec)]
return v
def norm(v):
n = math.sqrt(sum(x * x for x in v)) or 1.0
return [x / n for x in v]
class Corpus:
def __init__(self, rng):
self.rng = rng
self.topic = [unit(rng) for _ in range(TOPICS)]
self.code_dir = unit(rng)
self.vec = {}
self.docs = []
for i in range(DOCS):
t = i % TOPICS
words = ["t{}w{}".format(t, rng.randrange(40)) for _ in range(20)]
words += ["d{}x{}".format(i, k) for k in range(3)]
words += ["E{}".format(1000 + i)]
self.docs.append((t, words))
self.df = {}
for _, ws in self.docs:
for w in set(ws):
self.df[w] = self.df.get(w, 0) + 1
self.avg = sum(len(ws) for _, ws in self.docs) / DOCS
self.emb = [self.embed(ws) for _, ws in self.docs]
def wvec(self, w):
if w in self.vec:
return self.vec[w]
r = self.rng
if w.startswith("t"): # topic word or synonym
t = int(w[1:].replace("s", "w").split("w")[0])
v = mix((1.0, self.topic[t]), (0.5, unit(r)))
elif w.startswith("d"): # detail word/paraphrase
base = w.replace("p", "x")
if base not in self.vec:
self.vec[base] = unit(r)
v = mix((1.0, self.vec[base]), (0.3, unit(r)))
if w == base:
return self.vec[base]
elif w.startswith("E"): # code: almost one token
v = mix((1.0, self.code_dir), (0.15, unit(r)))
else:
v = unit(r)
self.vec[w] = v
return v
def embed(self, words):
return norm(mix(*[(1.0, self.wvec(w)) for w in words]))
def bm25(self, q):
out = []
for i, (_, ws) in enumerate(self.docs):
s = 0.0
for w in set(q):
f = ws.count(w)
if f:
n = self.df[w]
idf = math.log(1 + (DOCS - n + 0.5) / (n + 0.5))
s += idf * f * 2.2 / (f + 1.2 * (0.25 + 0.75 * len(ws)
/ self.avg))
out.append(s)
return out
def dense(self, q):
qv = self.embed(q)
return [sum(a * b for a, b in zip(qv, e)) for e in self.emb]
def ranks(scores, rng):
order = sorted(range(len(scores)),
key=lambda i: (-scores[i], rng.random()))
r = [0] * len(scores)
for pos, i in enumerate(order):
r[i] = pos + 1
return r
if __name__ == "__main__":
rng = random.Random(61004)
c = Corpus(rng)
res = {}
for qtype in ("paraphrase", "exact code"):
hits = {"BM25": [0, 0], "dense": [0, 0], "RRF-all": [0, 0],
"RRF-100": [0, 0]}
for _ in range(200):
g = rng.randrange(DOCS)
t = c.docs[g][0]
if qtype == "paraphrase":
q = ["t{}s{}".format(t, rng.randrange(40)) for _ in range(3)]
q += ["d{}p{}".format(g, k) for k in range(3)]
else:
q = ["what", "does", "code", "E{}".format(1000 + g), "mean"]
sb = c.bm25(q)
rb = ranks(sb, rng)
rd = ranks(c.dense(q), rng)
fused = [1 / (60 + a) + 1 / (60 + b) for a, b in zip(rb, rd)]
rf = ranks(fused, rng)
f100 = [(1 / (60 + a) if a <= 100 and s > 0 else 0.0)
+ (1 / (60 + b) if b <= 100 else 0.0)
for a, b, s in zip(rb, rd, sb)]
r100 = ranks(f100, rng)
for name, r in (("BM25", rb), ("dense", rd), ("RRF-all", rf),
("RRF-100", r100)):
hits[name][0] += r[g] == 1
hits[name][1] += r[g] <= 10
res[qtype] = hits
print(" query type ranker recall@1 recall@10")
for qtype, hits in res.items():
for name, (h1, h10) in hits.items():
print(" {:<13} {:<7} {:7.3f} {:7.3f}".format(
qtype, name, h1 / 200, h10 / 200))
print("\nReading: BM25 cannot find a chunk that shares no words with")
print("the query; the dense model cannot tell one code from another,")
print("because every code is nearly the same vector. Fusion is not")
print("automatically the best of both: fusing full rankings lets the")
print("blind ranker's noise outvote the one that knows. Fusing short")
print("lists, with zero-score chunks left out, repairs paraphrase")
print("queries completely but leaves exact-code queries a near coin")
print("toss: the dense top hit ties with the true chunk.")唯一数据源与引用
范雅福(Fan, Y.). (2026). 《检索增强生成几何、倒数秩融合与多智能体误差级联》第 2 章「词法与稠密检索:各自的盲区与倒数秩融合的平滑常数」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090170
DOI: 10.5281/zenodo.24090170 · 阅读本章所在专著 · 5DL-B-061 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026061,
author = {Fan, Yafu},
title = {Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.24090170},
url = {https://doi.org/10.5281/zenodo.24090170}
}