5D Library › Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades · 中文
Chapter 2. Lexical and Dense Retrieval: Their Blind Spots and the Smoothing Constant of Reciprocal Rank Fusion
Two retrievers, two failures
Sparse lexical retrieval, typified by BM25, scores exact overlap between query terms and text (Robertson and Zaragoza 2009); dense retrieval scores the similarity of semantic vectors (Karpukhin et al. 2020). The first knows the letters but not the meaning; the second knows the meaning but not the letters. On queries containing error codes, part numbers, version strings or rare terms, dense retrieval has a structural weakness: the tokenizer cuts these strings into common sub-words, the vectors of different codes are almost identical, and the inner product cannot tell them apart. This chapter first shows the two failures side by side on a synthetic corpus, then asks when the most common way of combining them — reciprocal rank fusion (RRF) — works, and what its smoothing constant should be.
Blind spots side by side
Calculation
# Chapter 2. p04: Lexical (BM25) and dense retrieval fail on different
# queries.
# Synthetic corpus: 2,000 chunks in 50 topics. Each chunk has 20 topic
# words, 3 detail words of its own, and a unique error code such as
# E1043. Two query types, 200 each:
# paraphrase 3 topic synonyms + 3 paraphrased detail words - no word
# in common with the gold chunk
# exact code "what does code E1043 mean" - the code is the only clue
# BM25 (k1 = 1.2, b = 0.75) scores exact token overlap. The dense model
# (d = 64) maps a word and its synonym/paraphrase to nearby vectors, but
# like a subword tokenizer sees every code as almost the same token: a
# shared "code" vector plus only 0.15 of a code-specific part. A chunk
# or query vector is the normalised mean of its word vectors.
# Hybrid: reciprocal rank fusion, score = sum 1 / (60 + rank), done two
# ways: RRF-all fuses complete rankings (ties broken at random); RRF-100
# fuses each ranker's top-100 list only, and BM25 lists only chunks with
# a positive score - the usual implementation.
import math
import random
D, TOPICS, DOCS = 64, 50, 2000
def unit(rng):
v = [rng.gauss(0, 1) for _ in range(D)]
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
def mix(*parts):
v = [0.0] * D
for w, vec in parts:
v = [a + w * b for a, b in zip(v, vec)]
return v
def norm(v):
n = math.sqrt(sum(x * x for x in v)) or 1.0
return [x / n for x in v]
class Corpus:
def __init__(self, rng):
self.rng = rng
self.topic = [unit(rng) for _ in range(TOPICS)]
self.code_dir = unit(rng)
self.vec = {}
self.docs = []
for i in range(DOCS):
t = i % TOPICS
words = ["t{}w{}".format(t, rng.randrange(40)) for _ in range(20)]
words += ["d{}x{}".format(i, k) for k in range(3)]
words += ["E{}".format(1000 + i)]
self.docs.append((t, words))
self.df = {}
for _, ws in self.docs:
for w in set(ws):
self.df[w] = self.df.get(w, 0) + 1
self.avg = sum(len(ws) for _, ws in self.docs) / DOCS
self.emb = [self.embed(ws) for _, ws in self.docs]
def wvec(self, w):
if w in self.vec:
return self.vec[w]
r = self.rng
if w.startswith("t"): # topic word or synonym
t = int(w[1:].replace("s", "w").split("w")[0])
v = mix((1.0, self.topic[t]), (0.5, unit(r)))
elif w.startswith("d"): # detail word/paraphrase
base = w.replace("p", "x")
if base not in self.vec:
self.vec[base] = unit(r)
v = mix((1.0, self.vec[base]), (0.3, unit(r)))
if w == base:
return self.vec[base]
elif w.startswith("E"): # code: almost one token
v = mix((1.0, self.code_dir), (0.15, unit(r)))
else:
v = unit(r)
self.vec[w] = v
return v
def embed(self, words):
return norm(mix(*[(1.0, self.wvec(w)) for w in words]))
def bm25(self, q):
out = []
for i, (_, ws) in enumerate(self.docs):
s = 0.0
for w in set(q):
f = ws.count(w)
if f:
n = self.df[w]
idf = math.log(1 + (DOCS - n + 0.5) / (n + 0.5))
s += idf * f * 2.2 / (f + 1.2 * (0.25 + 0.75 * len(ws)
/ self.avg))
out.append(s)
return out
def dense(self, q):
qv = self.embed(q)
return [sum(a * b for a, b in zip(qv, e)) for e in self.emb]
def ranks(scores, rng):
order = sorted(range(len(scores)),
key=lambda i: (-scores[i], rng.random()))
r = [0] * len(scores)
for pos, i in enumerate(order):
r[i] = pos + 1
return r
if __name__ == "__main__":
rng = random.Random(61004)
c = Corpus(rng)
res = {}
for qtype in ("paraphrase", "exact code"):
hits = {"BM25": [0, 0], "dense": [0, 0], "RRF-all": [0, 0],
"RRF-100": [0, 0]}
for _ in range(200):
g = rng.randrange(DOCS)
t = c.docs[g][0]
if qtype == "paraphrase":
q = ["t{}s{}".format(t, rng.randrange(40)) for _ in range(3)]
q += ["d{}p{}".format(g, k) for k in range(3)]
else:
q = ["what", "does", "code", "E{}".format(1000 + g), "mean"]
sb = c.bm25(q)
rb = ranks(sb, rng)
rd = ranks(c.dense(q), rng)
fused = [1 / (60 + a) + 1 / (60 + b) for a, b in zip(rb, rd)]
rf = ranks(fused, rng)
f100 = [(1 / (60 + a) if a <= 100 and s > 0 else 0.0)
+ (1 / (60 + b) if b <= 100 else 0.0)
for a, b, s in zip(rb, rd, sb)]
r100 = ranks(f100, rng)
for name, r in (("BM25", rb), ("dense", rd), ("RRF-all", rf),
("RRF-100", r100)):
hits[name][0] += r[g] == 1
hits[name][1] += r[g] <= 10
res[qtype] = hits
print(" query type ranker recall@1 recall@10")
for qtype, hits in res.items():
for name, (h1, h10) in hits.items():
print(" {:<13} {:<7} {:7.3f} {:7.3f}".format(
qtype, name, h1 / 200, h10 / 200))
print("\nReading: BM25 cannot find a chunk that shares no words with")
print("the query; the dense model cannot tell one code from another,")
print("because every code is nearly the same vector. Fusion is not")
print("automatically the best of both: fusing full rankings lets the")
print("blind ranker's noise outvote the one that knows. Fusing short")
print("lists, with zero-score chunks left out, repairs paraphrase")
print("queries completely but leaves exact-code queries a near coin")
print("toss: the dense top hit ties with the true chunk.")
# query type ranker recall@1 recall@10
# paraphrase BM25 0.005 0.005
# paraphrase dense 0.980 1.000
# paraphrase RRF-all 0.055 0.740
# paraphrase RRF-100 0.980 1.000
# exact code BM25 1.000 1.000
# exact code dense 0.000 0.005
# exact code RRF-all 0.055 0.750
# exact code RRF-100 0.475 1.000
#
# Reading: BM25 cannot find a chunk that shares no words with
# the query; the dense model cannot tell one code from another,
# because every code is nearly the same vector. Fusion is not
# automatically the best of both: fusing full rankings lets the
# blind ranker's noise outvote the one that knows. Fusing short
# lists, with zero-score chunks left out, repairs paraphrase
# queries completely but leaves exact-code queries a near coin
# toss: the dense top hit ties with the true chunk.On paraphrase queries (sharing not a single word with the gold chunk), BM25’s recall@1 is only 0.005 and dense retrieval’s 0.980. On exact-code queries it is the other way round: BM25 1.000, dense retrieval 0.000 — it sees every code as nearly the same vector, the code-specific part being a small fraction of the vector, drowned by topical differences. This is a direct demonstration of “a resolution floor for pure vector inner products on exact matches”.
Fusion does not automatically give the best of both. Fusing the two complete rankings directly (RRF-all) leaves both query types at a recall@1 of only 0.055: for each type one retriever is ranking at random, and its noise gets the same weight as the other’s correct signal. Fusing only each retriever’s top 100, with BM25’s zero-score chunks left out, as is usual (RRF-100), restores paraphrase queries completely (0.980) but takes exact-code queries only to 0.475: the wrong chunk dense retrieval ranks first ties in fused score with the right chunk BM25 ranks first — close to a coin toss. The 0.475 comes from a tie-breaking degeneracy at equal weight: the right chunk is hit by one list only (BM25’s first place), the wrong chunk by one list only (dense retrieval’s first place), both fuse to the same score , and which comes first depends only on how the tie is broken. Recall@10 is 1.000 for both, so the gold chunk is always in the top ten; the problem is first place only.
Note
What this means in practice. Before fusing two retrievers, look at each one’s blind spots by query type. For queries known to contain identifiers (detectable with a simple regular expression), giving lexical retrieval more weight or routing them to it directly often does better than fusing all queries alike.
The smoothing constant k
Calculation
Reciprocal rank fusion. Each chunk’s fused score is where is its rank under retriever (Cormack et al. 2009). With small , first and second place score very differently and the fusion is decided by each retriever’s head; with large , the score changes slowly with rank and the fusion approaches an average of ranks. Cormack and colleagues used in the original paper.
Calculation
# Chapter 2. p05: How the smoothing constant k in RRF moves head and tail.
# Two rankers score 1,000 chunks. Irrelevant chunks get standard-normal
# scores; the gold chunk gets mu plus noise. The two rankers' noise can
# be correlated (rho), as when both are fooled by the same chunk. Fused
# score = sum over rankers of 1 / (k + rank). Three scenarios, 2,000
# queries each:
# S1 equal mu = 2.5 and 2.5, rho = 0
# S2 unequal mu = 3.5 and 1.0, rho = 0 (one strong, one weak)
# S3 shared mu = 2.5 and 2.5, rho = 0.6
# For comparison, a convex blend of the raw scores, alpha * a + (1 -
# alpha) * b, with the best alpha on a 0.05 grid (it needs comparable
# scores, which real rankers do not give for free).
import math
import random
N, Q = 1000, 2000
KS = (1, 5, 20, 60, 200, 1000)
def scenario(rng, mu_a, mu_b, rho):
out = []
for _ in range(Q):
a, b = [], []
for _ in range(N):
z = rng.gauss(0, 1)
a.append(z)
b.append(rho * z + math.sqrt(1 - rho * rho) * rng.gauss(0, 1))
a[0] += mu_a
b[0] += mu_b
out.append((a, b))
return out
def rank_of_gold(score):
g = score[0]
return 1 + sum(1 for s in score[1:] if s > g)
def rank_list(score):
order = sorted(range(len(score)), key=lambda i: -score[i])
r = [0] * len(score)
for p, i in enumerate(order):
r[i] = p + 1
return r
def metrics(golds):
n = len(golds)
return (sum(r == 1 for r in golds) / n, sum(1 / r for r in golds) / n,
sum(r <= 20 for r in golds) / n)
if __name__ == "__main__":
rng = random.Random(61005)
for name, ma, mb, rho in (("S1 equal", 2.5, 2.5, 0.0),
("S2 unequal", 3.5, 1.0, 0.0),
("S3 shared", 2.5, 2.5, 0.6)):
data = scenario(rng, ma, mb, rho)
ranked = [(rank_list(a), rank_list(b), a, b) for a, b in data]
print("{} (alone: A r@1 {:.3f}, B r@1 {:.3f})".format(
name, metrics([ra[0] for ra, _, _, _ in ranked])[0],
metrics([rb[0] for _, rb, _, _ in ranked])[0]))
print(" k recall@1 MRR recall@20")
for k in KS:
g = [rank_of_gold([1 / (k + x) + 1 / (k + y)
for x, y in zip(ra, rb)])
for ra, rb, _, _ in ranked]
r1, mrr, r20 = metrics(g)
print(" {:5d} {:.3f} {:.3f} {:.3f}".format(
k, r1, mrr, r20))
best = max((metrics([rank_of_gold([al * x + (1 - al) * y
for x, y in zip(a, b)])
for _, _, a, b in ranked])[1], al)
for al in [i / 20 for i in range(21)])
print(" best score blend: alpha {:.2f}, MRR {:.3f}\n".format(
best[1], best[0]))
print("Reading: with two equal rankers fusion more than doubles")
print("recall@1, but k = 1 is clearly worse than k = 20 to 1000.")
print("With one strong and one weak ranker, every k does worse at the")
print("head than the strong ranker alone - small k loses least - and")
print("from k = 200 the tail collapses too. With shared errors fusion")
print("gains little whatever k is. In none of the three is k = 60 the")
print("best for recall@1; where scores are comparable, a tuned blend")
print("beats every k.")
# S1 equal (alone: A r@1 0.229, B r@1 0.247)
# k recall@1 MRR recall@20
# 1 0.425 0.516 0.847
# 5 0.507 0.588 0.855
# 20 0.586 0.679 0.881
# 60 0.560 0.667 0.917
# 200 0.547 0.650 0.911
# 1000 0.541 0.642 0.895
# best score blend: alpha 0.50, MRR 0.713
#
# S2 unequal (alone: A r@1 0.604, B r@1 0.017)
# k recall@1 MRR recall@20
# 1 0.478 0.601 0.892
# 5 0.470 0.601 0.897
# 20 0.297 0.486 0.907
# 60 0.211 0.353 0.903
# 200 0.197 0.297 0.640
# 1000 0.191 0.279 0.567
# best score blend: alpha 0.80, MRR 0.738
#
# S3 shared (alone: A r@1 0.243, B r@1 0.229)
# k recall@1 MRR recall@20
# 1 0.309 0.412 0.729
# 5 0.321 0.432 0.737
# 20 0.319 0.434 0.755
# 60 0.323 0.434 0.756
# 200 0.324 0.434 0.751
# 1000 0.324 0.433 0.748
# best score blend: alpha 0.55, MRR 0.443
#
# Reading: with two equal rankers fusion more than doubles
# recall@1, but k = 1 is clearly worse than k = 20 to 1000.
# With one strong and one weak ranker, every k does worse at the
# head than the strong ranker alone - small k loses least - and
# from k = 200 the tail collapses too. With shared errors fusion
# gains little whatever k is. In none of the three is k = 60 the
# best for recall@1; where scores are comparable, a tuned blend
# beats every k.The three scenarios give three different answers. With two equally good retrievers (S1), fusion lifts recall@1 from 0.229 and 0.247 for either alone to at most 0.586 (at ); is clearly worse (0.425), and between 20 and 1000 makes little difference. With one strong and one weak retriever (S2), every does worse at the head than the strong retriever alone (0.604): the smaller , the smaller the loss (0.478 at ), only 0.211 at , and from even recall@20 collapses (0.640). When the two retrievers make the same mistakes (S3), fusion gains almost nothing, whatever .
In none of the three scenarios is the best value for recall@1. If the two retrievers’ scores can be put on a common scale, a tuned convex blend of scores (best ) beats every in S1 and S2 (MRR 0.713 and 0.738). Hence the correction to the outline: there is no that is optimal for every distribution; it depends on the retrievers’ relative strength and on how correlated their errors are, and should be tuned on one’s own queries.
Hypothesis · not established
The author’s hypothesis (not yet tested): in a real hybrid retrieval system, tuning (or the weights) separately for queries with and without identifiers raises recall@1 by at least 3 percentage points over a global . What would show it wrong: on a pre-registered evaluation set, the upper end of the 95 % confidence interval for the paired difference between per-class tuning and global is below 3 points.
Summary
Lexical retrieval cannot recognise a paraphrase and dense retrieval cannot recognise an identifier; their blind spots are complementary, but fusion does not automatically take the best of each. Fusing complete rankings lets the blind side’s noise cancel the side that knows; fusing short lists with zero-score chunks removed repairs paraphrase queries but leaves identifier queries near a coin toss. RRF’s smoothing constant has no universal optimum: with comparable retrievers matters little from 20 up, with one strong and one weak no beats the strong one alone, and with correlated errors fusion barely helps.
References
Fan, Y. (2026). Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades: High-Dimensional Concentration of Measure, Hybrid Lexical-Dense Retrieval Bounds and Verifier-Gated Fixed Points [检索增强生成几何、倒数秩融合与多智能体误差级联] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090170 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).