What does RAG and Multi-Agent Reliability conclude, in numbers, about “The Routing of the *Qilue* and Han Fei's *Canyan*: Two Ancient Methods of Retrieval and Verification”?
Answer: Chapter 7 reads the Qilue as routing by category (routing must be more than 0.656 accurate to beat searching everything) and the Han Feizi's canyan as checking against several sources (sources sharing an origin are worth one however many).
Scope of the numbers: All data are synthetic, generated by the book's programs; the numbers show mechanisms and orders of magnitude and are not measurements of any product.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.656
| A. flat search over 60,000: top-1 0.286 | |
| search inside one division of 10,000: top-1 0.436 | |
| routing accuracy routed top-1 better than flat? | |
| 1.00 | 0.436 yes |
| 0.95 | 0.414 yes |
| 0.90 | 0.393 yes |
| 0.80 | 0.349 yes |
| 0.70 | 0.305 yes |
| 0.60 | 0.262 no |
| break-even routing accuracy | 0.656 |
| B. majority of k sources, each right with 0.8 | |
| k c | 0 c = 0.3 c = 0.6 |
| 1 | 0.8000 0.8000 0.8000 |
| 3 | 0.8960 0.8672 0.8384 |
| 5 | 0.9421 0.8995 0.8568 |
| 7 | 0.9667 0.9167 0.8667 |
| C. two collators, each catching 0.7 of errors | |
| shared misses c | 0.0: caught 0.910 |
| shared misses c | 0.3: caught 0.847 |
| shared misses c | 0.6: caught 0.784 |
| Reading: a catalogue helps retrieval only when its routing is | |
| more reliable than the break-even accuracy - a misfiled book is | |
| lost however good the shelf search. More sources raise | |
| accuracy only to the extent that they are independent; sources | |
| that copy one another add confidence without adding evidence. |
Show the full program
# Chapter 7. p15: Liu Xin's catalogue as routing; Han Fei's canyan as voting.
# A. The Qilue sorted the imperial library into six substantive
# divisions plus a summary. As retrieval: route a query to one of six
# divisions (right with probability a), then search only that
# division. Search uses the extreme-value model of p01 with
# effective dimension 150 and a relevant cosine of 0.30: top-1 is right
# when the relevant chunk beats the best of M distractors. Flat search
# over 60,000 chunks against routed search over 10,000.
# B. Canyan, "checking against several sources": k sources each right
# with 0.8, majority vote. With probability c all k merely repeat one
# source (shared provenance) - then k sources are worth one.
# C. Jiaochou, collation by two readers: each catches an error with 0.7;
# with probability c they miss the same errors together.
import math
import random
def erfinv(y):
x = 0.0 if abs(y) < 0.7 else math.copysign(
math.sqrt(-math.log((1 - abs(y)) / 2)), y)
for _ in range(60):
x -= (math.erf(x) - y) / (2 / math.sqrt(math.pi) * math.exp(-x * x))
return x
def top1(rng, m, d=150, trials=4000):
sd = 1 / math.sqrt(d)
ok = 0
for _ in range(trials):
rel = 0.30 + rng.gauss(0, sd)
u = rng.random() ** (1 / m)
ok += math.sqrt(2) * erfinv(2 * u - 1) * sd < rel
return ok / trials
def vote(k, p):
return sum(math.comb(k, j) * p ** j * (1 - p) ** (k - j)
for j in range(k // 2 + 1, k + 1))
if __name__ == "__main__":
rng = random.Random(61015)
flat = top1(rng, 60000)
inner = top1(rng, 10000)
print("A. flat search over 60,000: top-1 {:.3f}".format(flat))
print(" search inside one division of 10,000: top-1 {:.3f}".format(
inner))
print(" routing accuracy routed top-1 better than flat?")
for a in (1.0, 0.95, 0.9, 0.8, 0.7, 0.6):
r = a * inner
print(" {:.2f} {:.3f} {}".format(
a, r, "yes" if r > flat else "no"))
print(" break-even routing accuracy: {:.3f}".format(flat / inner))
print("\nB. majority of k sources, each right with 0.8")
print(" k c = 0 c = 0.3 c = 0.6")
for k in (1, 3, 5, 7):
print(" {} {:.4f} {:.4f} {:.4f}".format(
k, *[c * 0.8 + (1 - c) * vote(k, 0.8) for c in (0, 0.3, 0.6)]))
print("\nC. two collators, each catching 0.7 of errors")
for c in (0.0, 0.3, 0.6):
print(" shared misses c = {}: caught {:.3f}".format(
c, c * 0.7 + (1 - c) * (1 - 0.3 ** 2)))
print("\nReading: a catalogue helps retrieval only when its routing is")
print("more reliable than the break-even accuracy - a misfiled book is")
print("lost however good the shelf search. More sources raise")
print("accuracy only to the extent that they are independent; sources")
print("that copy one another add confidence without adding evidence.")Single source and citation
Fan, Y. (2026). Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades (Version 1.0.0), Chapter 7: The Routing of the *Qilue* and Han Fei's *Canyan*: Two Ancient Methods of Retrieval and Verification. 5D Library. https://doi.org/10.5281/zenodo.24090170
DOI: 10.5281/zenodo.24090170 · Read the monograph · 5DL-B-061 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026061,
author = {Fan, Yafu},
title = {Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.24090170},
url = {https://doi.org/10.5281/zenodo.24090170}
}Other chapter answers in this book
- Chapter 1 · High-Dimensional Geometry: Concentration of Measure, Effective Dimension and Hub Chunks
- Chapter 2 · Lexical and Dense Retrieval: Their Blind Spots and the Smoothing Constant of Reciprocal Rank Fusion
- Chapter 3 · The Context Window: Position Decay, Evidence Dilution and the Best Number of Chunks
- Chapter 4 · Multi-Agent Pipelines: Error Cascades and the Verifier-Gated Absorbing Markov Chain
- Chapter 5 · Verifier Independence and the Cost of Retries
- Chapter 6 · The Statistics of Evaluation Itself: Sample Size, Judge Bias and the Winner's Curse
- Chapter 8 · A Five-Step Check and a Pre-Registered Offline Evaluation