关于「词法与稠密检索:各自的盲区与倒数秩融合的平滑常数」,《检索增强与多智能体的可靠性》给出的定量结论是什么?

结论:第二章说明稠密检索认不出编号、词法检索认不出改写,融合完整排名时两类查询的 recall@1 都只剩 0.055,只融合短列表时编号查询停在 0.475;倒数秩融合的平滑常数 k = 60 在三种情形下都不是 recall@1 的最优值,一强一弱时任何 k 都不如强检索器单用(0.604)。

数据口径:全部数据为程序生成的合成数据,数字说明机制与量级,不是对任何产品的测量。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.055, 0.475

   query type     ranker   recall@1   recall@10
   paraphrase     BM25       0.005      0.005
   paraphrase     dense      0.980      1.000
   paraphrase     RRF-all    0.055      0.740
   paraphrase     RRF-100    0.980      1.000
   exact code     BM25       1.000      1.000
   exact code     dense      0.000      0.005
   exact code     RRF-all    0.055      0.750
   exact code     RRF-100    0.475      1.000
Reading: BM25 cannot find a chunk that shares no words with
the query; the dense model cannot tell one code from another,
because every code is nearly the same vector. Fusion is not
automatically the best of both: fusing full rankings lets the
blind ranker's noise outvote the one that knows. Fusing short
lists, with zero-score chunks left out, repairs paraphrase
queries completely but leaves exact-code queries a near coin
toss: the dense top hit ties with the true chunk.

核心公式

\[\mathrm{RRF}(d) = \sum_{i} \frac{1}{k + r_i(d)} ,\]
查看完整验算程序
# Chapter 2. p04: Lexical (BM25) and dense retrieval fail on different
# queries.
# Synthetic corpus: 2,000 chunks in 50 topics. Each chunk has 20 topic
# words, 3 detail words of its own, and a unique error code such as
# E1043. Two query types, 200 each:
#   paraphrase  3 topic synonyms + 3 paraphrased detail words - no word
#               in common with the gold chunk
#   exact code  "what does code E1043 mean" - the code is the only clue
# BM25 (k1 = 1.2, b = 0.75) scores exact token overlap. The dense model
# (d = 64) maps a word and its synonym/paraphrase to nearby vectors, but
# like a subword tokenizer sees every code as almost the same token: a
# shared "code" vector plus only 0.15 of a code-specific part. A chunk
# or query vector is the normalised mean of its word vectors.
# Hybrid: reciprocal rank fusion, score = sum 1 / (60 + rank), done two
# ways: RRF-all fuses complete rankings (ties broken at random); RRF-100
# fuses each ranker's top-100 list only, and BM25 lists only chunks with
# a positive score - the usual implementation.
import math
import random

D, TOPICS, DOCS = 64, 50, 2000

def unit(rng):
    v = [rng.gauss(0, 1) for _ in range(D)]
    n = math.sqrt(sum(x * x for x in v))
    return [x / n for x in v]

def mix(*parts):
    v = [0.0] * D
    for w, vec in parts:
        v = [a + w * b for a, b in zip(v, vec)]
    return v

def norm(v):
    n = math.sqrt(sum(x * x for x in v)) or 1.0
    return [x / n for x in v]

class Corpus:
    def __init__(self, rng):
        self.rng = rng
        self.topic = [unit(rng) for _ in range(TOPICS)]
        self.code_dir = unit(rng)
        self.vec = {}
        self.docs = []
        for i in range(DOCS):
            t = i % TOPICS
            words = ["t{}w{}".format(t, rng.randrange(40)) for _ in range(20)]
            words += ["d{}x{}".format(i, k) for k in range(3)]
            words += ["E{}".format(1000 + i)]
            self.docs.append((t, words))
        self.df = {}
        for _, ws in self.docs:
            for w in set(ws):
                self.df[w] = self.df.get(w, 0) + 1
        self.avg = sum(len(ws) for _, ws in self.docs) / DOCS
        self.emb = [self.embed(ws) for _, ws in self.docs]

    def wvec(self, w):
        if w in self.vec:
            return self.vec[w]
        r = self.rng
        if w.startswith("t"):                       # topic word or synonym
            t = int(w[1:].replace("s", "w").split("w")[0])
            v = mix((1.0, self.topic[t]), (0.5, unit(r)))
        elif w.startswith("d"):                     # detail word/paraphrase
            base = w.replace("p", "x")
            if base not in self.vec:
                self.vec[base] = unit(r)
            v = mix((1.0, self.vec[base]), (0.3, unit(r)))
            if w == base:
                return self.vec[base]
        elif w.startswith("E"):                     # code: almost one token
            v = mix((1.0, self.code_dir), (0.15, unit(r)))
        else:
            v = unit(r)
        self.vec[w] = v
        return v

    def embed(self, words):
        return norm(mix(*[(1.0, self.wvec(w)) for w in words]))

    def bm25(self, q):
        out = []
        for i, (_, ws) in enumerate(self.docs):
            s = 0.0
            for w in set(q):
                f = ws.count(w)
                if f:
                    n = self.df[w]
                    idf = math.log(1 + (DOCS - n + 0.5) / (n + 0.5))
                    s += idf * f * 2.2 / (f + 1.2 * (0.25 + 0.75 * len(ws)
                                                     / self.avg))
            out.append(s)
        return out

    def dense(self, q):
        qv = self.embed(q)
        return [sum(a * b for a, b in zip(qv, e)) for e in self.emb]

def ranks(scores, rng):
    order = sorted(range(len(scores)),
                   key=lambda i: (-scores[i], rng.random()))
    r = [0] * len(scores)
    for pos, i in enumerate(order):
        r[i] = pos + 1
    return r

if __name__ == "__main__":
    rng = random.Random(61004)
    c = Corpus(rng)
    res = {}
    for qtype in ("paraphrase", "exact code"):
        hits = {"BM25": [0, 0], "dense": [0, 0], "RRF-all": [0, 0],
                "RRF-100": [0, 0]}
        for _ in range(200):
            g = rng.randrange(DOCS)
            t = c.docs[g][0]
            if qtype == "paraphrase":
                q = ["t{}s{}".format(t, rng.randrange(40)) for _ in range(3)]
                q += ["d{}p{}".format(g, k) for k in range(3)]
            else:
                q = ["what", "does", "code", "E{}".format(1000 + g), "mean"]
            sb = c.bm25(q)
            rb = ranks(sb, rng)
            rd = ranks(c.dense(q), rng)
            fused = [1 / (60 + a) + 1 / (60 + b) for a, b in zip(rb, rd)]
            rf = ranks(fused, rng)
            f100 = [(1 / (60 + a) if a <= 100 and s > 0 else 0.0)
                    + (1 / (60 + b) if b <= 100 else 0.0)
                    for a, b, s in zip(rb, rd, sb)]
            r100 = ranks(f100, rng)
            for name, r in (("BM25", rb), ("dense", rd), ("RRF-all", rf),
                            ("RRF-100", r100)):
                hits[name][0] += r[g] == 1
                hits[name][1] += r[g] <= 10
        res[qtype] = hits
    print("   query type     ranker   recall@1   recall@10")
    for qtype, hits in res.items():
        for name, (h1, h10) in hits.items():
            print("   {:<13}  {:<7}  {:7.3f}    {:7.3f}".format(
                qtype, name, h1 / 200, h10 / 200))
    print("\nReading: BM25 cannot find a chunk that shares no words with")
    print("the query; the dense model cannot tell one code from another,")
    print("because every code is nearly the same vector. Fusion is not")
    print("automatically the best of both: fusing full rankings lets the")
    print("blind ranker's noise outvote the one that knows. Fusing short")
    print("lists, with zero-score chunks left out, repairs paraphrase")
    print("queries completely but leaves exact-code queries a near coin")
    print("toss: the dense top hit ties with the true chunk.")

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《检索增强生成几何、倒数秩融合与多智能体误差级联》第 2 章「词法与稠密检索:各自的盲区与倒数秩融合的平滑常数」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090170

DOI: 10.5281/zenodo.24090170 · 阅读本章所在专著 · 5DL-B-061 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026061,
  author    = {Fan, Yafu},
  title     = {Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.24090170},
  url       = {https://doi.org/10.5281/zenodo.24090170}
}

本书其他章节结论