5D Library › Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades · 中文

Chapter 2. Lexical and Dense Retrieval: Their Blind Spots and the Smoothing Constant of Reciprocal Rank Fusion

Two retrievers, two failures

Sparse lexical retrieval, typified by BM25, scores exact overlap between query terms and text (Robertson and Zaragoza 2009); dense retrieval scores the similarity of semantic vectors (Karpukhin et al. 2020). The first knows the letters but not the meaning; the second knows the meaning but not the letters. On queries containing error codes, part numbers, version strings or rare terms, dense retrieval has a structural weakness: the tokenizer cuts these strings into common sub-words, the vectors of different codes are almost identical, and the inner product cannot tell them apart. This chapter first shows the two failures side by side on a synthetic corpus, then asks when the most common way of combining them — reciprocal rank fusion (RRF) — works, and what its smoothing constant kk should be.

Blind spots side by side

Calculation

# Chapter 2. p04: Lexical (BM25) and dense retrieval fail on different
# queries.
# Synthetic corpus: 2,000 chunks in 50 topics. Each chunk has 20 topic
# words, 3 detail words of its own, and a unique error code such as
# E1043. Two query types, 200 each:
#   paraphrase  3 topic synonyms + 3 paraphrased detail words - no word
#               in common with the gold chunk
#   exact code  "what does code E1043 mean" - the code is the only clue
# BM25 (k1 = 1.2, b = 0.75) scores exact token overlap. The dense model
# (d = 64) maps a word and its synonym/paraphrase to nearby vectors, but
# like a subword tokenizer sees every code as almost the same token: a
# shared "code" vector plus only 0.15 of a code-specific part. A chunk
# or query vector is the normalised mean of its word vectors.
# Hybrid: reciprocal rank fusion, score = sum 1 / (60 + rank), done two
# ways: RRF-all fuses complete rankings (ties broken at random); RRF-100
# fuses each ranker's top-100 list only, and BM25 lists only chunks with
# a positive score - the usual implementation.
import math
import random

D, TOPICS, DOCS = 64, 50, 2000

def unit(rng):
    v = [rng.gauss(0, 1) for _ in range(D)]
    n = math.sqrt(sum(x * x for x in v))
    return [x / n for x in v]

def mix(*parts):
    v = [0.0] * D
    for w, vec in parts:
        v = [a + w * b for a, b in zip(v, vec)]
    return v

def norm(v):
    n = math.sqrt(sum(x * x for x in v)) or 1.0
    return [x / n for x in v]

class Corpus:
    def __init__(self, rng):
        self.rng = rng
        self.topic = [unit(rng) for _ in range(TOPICS)]
        self.code_dir = unit(rng)
        self.vec = {}
        self.docs = []
        for i in range(DOCS):
            t = i % TOPICS
            words = ["t{}w{}".format(t, rng.randrange(40)) for _ in range(20)]
            words += ["d{}x{}".format(i, k) for k in range(3)]
            words += ["E{}".format(1000 + i)]
            self.docs.append((t, words))
        self.df = {}
        for _, ws in self.docs:
            for w in set(ws):
                self.df[w] = self.df.get(w, 0) + 1
        self.avg = sum(len(ws) for _, ws in self.docs) / DOCS
        self.emb = [self.embed(ws) for _, ws in self.docs]

    def wvec(self, w):
        if w in self.vec:
            return self.vec[w]
        r = self.rng
        if w.startswith("t"):                       # topic word or synonym
            t = int(w[1:].replace("s", "w").split("w")[0])
            v = mix((1.0, self.topic[t]), (0.5, unit(r)))
        elif w.startswith("d"):                     # detail word/paraphrase
            base = w.replace("p", "x")
            if base not in self.vec:
                self.vec[base] = unit(r)
            v = mix((1.0, self.vec[base]), (0.3, unit(r)))
            if w == base:
                return self.vec[base]
        elif w.startswith("E"):                     # code: almost one token
            v = mix((1.0, self.code_dir), (0.15, unit(r)))
        else:
            v = unit(r)
        self.vec[w] = v
        return v

    def embed(self, words):
        return norm(mix(*[(1.0, self.wvec(w)) for w in words]))

    def bm25(self, q):
        out = []
        for i, (_, ws) in enumerate(self.docs):
            s = 0.0
            for w in set(q):
                f = ws.count(w)
                if f:
                    n = self.df[w]
                    idf = math.log(1 + (DOCS - n + 0.5) / (n + 0.5))
                    s += idf * f * 2.2 / (f + 1.2 * (0.25 + 0.75 * len(ws)
                                                     / self.avg))
            out.append(s)
        return out

    def dense(self, q):
        qv = self.embed(q)
        return [sum(a * b for a, b in zip(qv, e)) for e in self.emb]

def ranks(scores, rng):
    order = sorted(range(len(scores)),
                   key=lambda i: (-scores[i], rng.random()))
    r = [0] * len(scores)
    for pos, i in enumerate(order):
        r[i] = pos + 1
    return r

if __name__ == "__main__":
    rng = random.Random(61004)
    c = Corpus(rng)
    res = {}
    for qtype in ("paraphrase", "exact code"):
        hits = {"BM25": [0, 0], "dense": [0, 0], "RRF-all": [0, 0],
                "RRF-100": [0, 0]}
        for _ in range(200):
            g = rng.randrange(DOCS)
            t = c.docs[g][0]
            if qtype == "paraphrase":
                q = ["t{}s{}".format(t, rng.randrange(40)) for _ in range(3)]
                q += ["d{}p{}".format(g, k) for k in range(3)]
            else:
                q = ["what", "does", "code", "E{}".format(1000 + g), "mean"]
            sb = c.bm25(q)
            rb = ranks(sb, rng)
            rd = ranks(c.dense(q), rng)
            fused = [1 / (60 + a) + 1 / (60 + b) for a, b in zip(rb, rd)]
            rf = ranks(fused, rng)
            f100 = [(1 / (60 + a) if a <= 100 and s > 0 else 0.0)
                    + (1 / (60 + b) if b <= 100 else 0.0)
                    for a, b, s in zip(rb, rd, sb)]
            r100 = ranks(f100, rng)
            for name, r in (("BM25", rb), ("dense", rd), ("RRF-all", rf),
                            ("RRF-100", r100)):
                hits[name][0] += r[g] == 1
                hits[name][1] += r[g] <= 10
        res[qtype] = hits
    print("   query type     ranker   recall@1   recall@10")
    for qtype, hits in res.items():
        for name, (h1, h10) in hits.items():
            print("   {:<13}  {:<7}  {:7.3f}    {:7.3f}".format(
                qtype, name, h1 / 200, h10 / 200))
    print("\nReading: BM25 cannot find a chunk that shares no words with")
    print("the query; the dense model cannot tell one code from another,")
    print("because every code is nearly the same vector. Fusion is not")
    print("automatically the best of both: fusing full rankings lets the")
    print("blind ranker's noise outvote the one that knows. Fusing short")
    print("lists, with zero-score chunks left out, repairs paraphrase")
    print("queries completely but leaves exact-code queries a near coin")
    print("toss: the dense top hit ties with the true chunk.")
#    query type     ranker   recall@1   recall@10
#    paraphrase     BM25       0.005      0.005
#    paraphrase     dense      0.980      1.000
#    paraphrase     RRF-all    0.055      0.740
#    paraphrase     RRF-100    0.980      1.000
#    exact code     BM25       1.000      1.000
#    exact code     dense      0.000      0.005
#    exact code     RRF-all    0.055      0.750
#    exact code     RRF-100    0.475      1.000
#
# Reading: BM25 cannot find a chunk that shares no words with
# the query; the dense model cannot tell one code from another,
# because every code is nearly the same vector. Fusion is not
# automatically the best of both: fusing full rankings lets the
# blind ranker's noise outvote the one that knows. Fusing short
# lists, with zero-score chunks left out, repairs paraphrase
# queries completely but leaves exact-code queries a near coin
# toss: the dense top hit ties with the true chunk.

On paraphrase queries (sharing not a single word with the gold chunk), BM25’s recall@1 is only 0.005 and dense retrieval’s 0.980. On exact-code queries it is the other way round: BM25 1.000, dense retrieval 0.000 — it sees every code as nearly the same vector, the code-specific part being a small fraction of the vector, drowned by topical differences. This is a direct demonstration of “a resolution floor for pure vector inner products on exact matches”.

Fusion does not automatically give the best of both. Fusing the two complete rankings directly (RRF-all) leaves both query types at a recall@1 of only 0.055: for each type one retriever is ranking at random, and its noise gets the same weight as the other’s correct signal. Fusing only each retriever’s top 100, with BM25’s zero-score chunks left out, as is usual (RRF-100), restores paraphrase queries completely (0.980) but takes exact-code queries only to 0.475: the wrong chunk dense retrieval ranks first ties in fused score with the right chunk BM25 ranks first — close to a coin toss. The 0.475 comes from a tie-breaking degeneracy at equal weight: the right chunk is hit by one list only (BM25’s first place), the wrong chunk by one list only (dense retrieval’s first place), both fuse to the same score 1/(k+1)1/(k+1), and which comes first depends only on how the tie is broken. Recall@10 is 1.000 for both, so the gold chunk is always in the top ten; the problem is first place only.

Note

What this means in practice. Before fusing two retrievers, look at each one’s blind spots by query type. For queries known to contain identifiers (detectable with a simple regular expression), giving lexical retrieval more weight or routing them to it directly often does better than fusing all queries alike.

The smoothing constant k

Calculation

Reciprocal rank fusion. Each chunk’s fused score is RRF(d)=∑i1k+ri(d),\mathrm{RRF}(d) = \sum_{i} \frac{1}{k + r_i(d)} , where ri(d)r_i(d) is its rank under retriever ii (Cormack et al. 2009). With small kk, first and second place score very differently and the fusion is decided by each retriever’s head; with large kk, the score changes slowly with rank and the fusion approaches an average of ranks. Cormack and colleagues used k=60k = 60 in the original paper.

Calculation

# Chapter 2. p05: How the smoothing constant k in RRF moves head and tail.
# Two rankers score 1,000 chunks. Irrelevant chunks get standard-normal
# scores; the gold chunk gets mu plus noise. The two rankers' noise can
# be correlated (rho), as when both are fooled by the same chunk. Fused
# score = sum over rankers of 1 / (k + rank). Three scenarios, 2,000
# queries each:
#   S1 equal     mu = 2.5 and 2.5, rho = 0
#   S2 unequal   mu = 3.5 and 1.0, rho = 0   (one strong, one weak)
#   S3 shared    mu = 2.5 and 2.5, rho = 0.6
# For comparison, a convex blend of the raw scores, alpha * a + (1 -
# alpha) * b, with the best alpha on a 0.05 grid (it needs comparable
# scores, which real rankers do not give for free).
import math
import random

N, Q = 1000, 2000
KS = (1, 5, 20, 60, 200, 1000)

def scenario(rng, mu_a, mu_b, rho):
    out = []
    for _ in range(Q):
        a, b = [], []
        for _ in range(N):
            z = rng.gauss(0, 1)
            a.append(z)
            b.append(rho * z + math.sqrt(1 - rho * rho) * rng.gauss(0, 1))
        a[0] += mu_a
        b[0] += mu_b
        out.append((a, b))
    return out

def rank_of_gold(score):
    g = score[0]
    return 1 + sum(1 for s in score[1:] if s > g)

def rank_list(score):
    order = sorted(range(len(score)), key=lambda i: -score[i])
    r = [0] * len(score)
    for p, i in enumerate(order):
        r[i] = p + 1
    return r

def metrics(golds):
    n = len(golds)
    return (sum(r == 1 for r in golds) / n, sum(1 / r for r in golds) / n,
            sum(r <= 20 for r in golds) / n)

if __name__ == "__main__":
    rng = random.Random(61005)
    for name, ma, mb, rho in (("S1 equal", 2.5, 2.5, 0.0),
                              ("S2 unequal", 3.5, 1.0, 0.0),
                              ("S3 shared", 2.5, 2.5, 0.6)):
        data = scenario(rng, ma, mb, rho)
        ranked = [(rank_list(a), rank_list(b), a, b) for a, b in data]
        print("{}  (alone: A r@1 {:.3f}, B r@1 {:.3f})".format(
            name, metrics([ra[0] for ra, _, _, _ in ranked])[0],
            metrics([rb[0] for _, rb, _, _ in ranked])[0]))
        print("      k     recall@1    MRR     recall@20")
        for k in KS:
            g = [rank_of_gold([1 / (k + x) + 1 / (k + y)
                               for x, y in zip(ra, rb)])
                 for ra, rb, _, _ in ranked]
            r1, mrr, r20 = metrics(g)
            print("   {:5d}     {:.3f}     {:.3f}     {:.3f}".format(
                k, r1, mrr, r20))
        best = max((metrics([rank_of_gold([al * x + (1 - al) * y
                                           for x, y in zip(a, b)])
                             for _, _, a, b in ranked])[1], al)
                   for al in [i / 20 for i in range(21)])
        print("   best score blend: alpha {:.2f}, MRR {:.3f}\n".format(
            best[1], best[0]))
    print("Reading: with two equal rankers fusion more than doubles")
    print("recall@1, but k = 1 is clearly worse than k = 20 to 1000.")
    print("With one strong and one weak ranker, every k does worse at the")
    print("head than the strong ranker alone - small k loses least - and")
    print("from k = 200 the tail collapses too. With shared errors fusion")
    print("gains little whatever k is. In none of the three is k = 60 the")
    print("best for recall@1; where scores are comparable, a tuned blend")
    print("beats every k.")
# S1 equal  (alone: A r@1 0.229, B r@1 0.247)
#       k     recall@1    MRR     recall@20
#        1     0.425     0.516     0.847
#        5     0.507     0.588     0.855
#       20     0.586     0.679     0.881
#       60     0.560     0.667     0.917
#      200     0.547     0.650     0.911
#     1000     0.541     0.642     0.895
#    best score blend: alpha 0.50, MRR 0.713
#
# S2 unequal  (alone: A r@1 0.604, B r@1 0.017)
#       k     recall@1    MRR     recall@20
#        1     0.478     0.601     0.892
#        5     0.470     0.601     0.897
#       20     0.297     0.486     0.907
#       60     0.211     0.353     0.903
#      200     0.197     0.297     0.640
#     1000     0.191     0.279     0.567
#    best score blend: alpha 0.80, MRR 0.738
#
# S3 shared  (alone: A r@1 0.243, B r@1 0.229)
#       k     recall@1    MRR     recall@20
#        1     0.309     0.412     0.729
#        5     0.321     0.432     0.737
#       20     0.319     0.434     0.755
#       60     0.323     0.434     0.756
#      200     0.324     0.434     0.751
#     1000     0.324     0.433     0.748
#    best score blend: alpha 0.55, MRR 0.443
#
# Reading: with two equal rankers fusion more than doubles
# recall@1, but k = 1 is clearly worse than k = 20 to 1000.
# With one strong and one weak ranker, every k does worse at the
# head than the strong ranker alone - small k loses least - and
# from k = 200 the tail collapses too. With shared errors fusion
# gains little whatever k is. In none of the three is k = 60 the
# best for recall@1; where scores are comparable, a tuned blend
# beats every k.

The three scenarios give three different answers. With two equally good retrievers (S1), fusion lifts recall@1 from 0.229 and 0.247 for either alone to at most 0.586 (at k=20k = 20); k=1k = 1 is clearly worse (0.425), and kk between 20 and 1000 makes little difference. With one strong and one weak retriever (S2), every kk does worse at the head than the strong retriever alone (0.604): the smaller kk, the smaller the loss (0.478 at k=1k = 1), only 0.211 at k=60k = 60, and from k=200k = 200 even recall@20 collapses (0.640). When the two retrievers make the same mistakes (S3), fusion gains almost nothing, whatever kk.

In none of the three scenarios is k=60k = 60 the best value for recall@1. If the two retrievers’ scores can be put on a common scale, a tuned convex blend of scores (best α\alpha) beats every kk in S1 and S2 (MRR 0.713 and 0.738). Hence the correction to the outline: there is no kk that is optimal for every distribution; it depends on the retrievers’ relative strength and on how correlated their errors are, and should be tuned on one’s own queries.

Recall@1 of reciprocal rank fusion against the smoothing constant k, in three scenarios: two equally good retrievers, one strong and one weak, and correlated errors.

Hypothesis · not established

The author’s hypothesis (not yet tested): in a real hybrid retrieval system, tuning kk (or the weights) separately for queries with and without identifiers raises recall@1 by at least 3 percentage points over a global k=60k = 60. What would show it wrong: on a pre-registered evaluation set, the upper end of the 95 % confidence interval for the paired difference between per-class tuning and global k=60k = 60 is below 3 points.

Summary

Lexical retrieval cannot recognise a paraphrase and dense retrieval cannot recognise an identifier; their blind spots are complementary, but fusion does not automatically take the best of each. Fusing complete rankings lets the blind side’s noise cancel the side that knows; fusing short lists with zero-score chunks removed repairs paraphrase queries but leaves identifier queries near a coin toss. RRF’s smoothing constant has no universal optimum: with comparable retrievers kk matters little from 20 up, with one strong and one weak no kk beats the strong one alone, and with correlated errors fusion barely helps.

References

Cormack, Gordon V., Charles L. A. Clarke, and Stefan Büttcher. 2009. “Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods.” Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 758–59.
Karpukhin, Vladimir, Barlas Oğuz, Sewon Min, et al. 2020. “Dense Passage Retrieval for Open-Domain Question Answering.” Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 6769–81.
Robertson, Stephen, and Hugo Zaragoza. 2009. “The Probabilistic Relevance Framework: BM25 and Beyond.” Foundations and Trends in Information Retrieval 3 (4): 333–89.

Fan, Y. (2026). Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades: High-Dimensional Concentration of Measure, Hybrid Lexical-Dense Retrieval Bounds and Verifier-Gated Fixed Points [检索增强生成几何、倒数秩融合与多智能体误差级联] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090170 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).