What does RAG and Multi-Agent Reliability conclude, in numbers, about “High-Dimensional Geometry: Concentration of Measure, Effective Dimension and Hub Chunks”?
Answer: Chapter 1 uses concentration of measure, effective dimension and hubness to show that the larger the corpus, the higher the top-1 error (from 0.0730 to 0.5030 at 256 dimensions); that with 768 nominal dimensions and a power-law spectrum only 31.7 effective dimensions remain, putting the wrong chunk first among a hundred thousand with probability 0.9875; and that in a hundred dimensions 1 % of hub chunks intrude on the top ten of 77.7 % of unrelated questions.
Scope of the numbers: All data are synthetic, generated by the book's programs; the numbers show mechanisms and orders of magnitude and are not measurements of any product.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.0730, 0.5030, 256, 768
| A. cosine of random unit vectors | |
| d sd (simulated) | 1/sqrt(d) largest |cos| of 2,000 |
| 16 | 0.2524 0.2500 0.8060 |
| 64 | 0.1243 0.1250 0.4137 |
| 256 | 0.0623 0.0625 0.1969 |
| 1024 | 0.0320 0.0312 0.1029 |
| B. P(top-1 wrong), relevant cosine 0.30 (+ noise of same sd) | |
| d M | 1000 M = 10000 M = 100000 M = 1000000 |
| 64 | 0.7890 0.9270 0.9710 0.9850 |
| 256 | 0.0730 0.1740 0.3280 0.5030 |
| 768 | 0.0000 0.0000 0.0000 0.0010 |
| d | 64: sqrt(2 ln M / d) at M = 1e6 is 0.657 |
| d | 768: sqrt(2 ln M / d) at M = 1e6 is 0.190 |
| Reading: in this isotropic model more dimensions help - noise | |
| cosines shrink like 1/sqrt(d) - but every tenfold growth of the | |
| corpus raises the best distractor by a fixed step, so top-1 | |
| errors climb with M at any d. The next program shows | |
| why real embeddings behave as if d were far smaller. |
Key formula
Show the full program
# Chapter 1. p01: Concentration of measure and the corpus-size penalty.
# A. Cosines between independent random unit vectors in d dimensions
# have mean 0 and standard deviation 1/sqrt(d) (2,000 pairs each).
# B. Retrieval as an extreme-value race. The relevant chunk has cosine
# s = 0.30 with the query plus noise; each of M irrelevant chunks has a
# cosine that is pure noise of sd 1/sqrt(d) (isotropic model). Top-1 is
# wrong when the largest of M noise cosines beats the relevant one.
# The largest of M grows like sqrt(2 ln M / d). 1,000 queries per cell.
import math
import random
def unit(rng, d):
v = [rng.gauss(0, 1) for _ in range(d)]
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
def erfinv(y):
# Newton on erf(x) = y, from a good start
x = 0.0 if abs(y) < 0.7 else math.copysign(
math.sqrt(-math.log((1 - abs(y)) / 2)), y)
for _ in range(60):
err = math.erf(x) - y
x -= err / (2 / math.sqrt(math.pi) * math.exp(-x * x))
return x
if __name__ == "__main__":
rng = random.Random(61001)
print("A. cosine of random unit vectors")
print(" d sd (simulated) 1/sqrt(d) largest |cos| of 2,000")
for d in (16, 64, 256, 1024):
cs = []
for _ in range(2000):
a, b = unit(rng, d), unit(rng, d)
cs.append(sum(x * y for x, y in zip(a, b)))
sd = math.sqrt(sum(c * c for c in cs) / len(cs))
print(" {:4d} {:.4f} {:.4f} {:.4f}".format(
d, sd, 1 / math.sqrt(d), max(abs(c) for c in cs)))
print("\nB. P(top-1 wrong), relevant cosine 0.30 (+ noise of same sd)")
sizes = (1000, 10000, 100000, 1000000)
print(" d " + "".join(" M = {:<8d}".format(m) for m in sizes))
for d in (64, 256, 768):
sd = 1 / math.sqrt(d)
cells = []
for m in sizes:
wrong = 0
for _ in range(1000):
rel = 0.30 + rng.gauss(0, sd)
# max of m iid N(0, sd^2) by inverse CDF of the max
u = rng.random() ** (1 / m)
z = math.sqrt(2) * erfinv(2 * u - 1)
wrong += z * sd > rel
cells.append(" {:9.4f} ".format(wrong / 1000))
print(" {:4d}".format(d) + "".join(cells))
for d in (64, 768):
print(" d = {:3d}: sqrt(2 ln M / d) at M = 1e6 is {:.3f}".format(
d, math.sqrt(2 * math.log(1e6) / d)))
print("\nReading: in this isotropic model more dimensions help - noise")
print("cosines shrink like 1/sqrt(d) - but every tenfold growth of the")
print("corpus raises the best distractor by a fixed step, so top-1")
print("errors climb with M at any d. The next program shows")
print("why real embeddings behave as if d were far smaller.")Single source and citation
Fan, Y. (2026). Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades (Version 1.0.0), Chapter 1: High-Dimensional Geometry: Concentration of Measure, Effective Dimension and Hub Chunks. 5D Library. https://doi.org/10.5281/zenodo.24090170
DOI: 10.5281/zenodo.24090170 · Read the monograph · 5DL-B-061 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026061,
author = {Fan, Yafu},
title = {Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.24090170},
url = {https://doi.org/10.5281/zenodo.24090170}
}Other chapter answers in this book
- Chapter 2 · Lexical and Dense Retrieval: Their Blind Spots and the Smoothing Constant of Reciprocal Rank Fusion
- Chapter 3 · The Context Window: Position Decay, Evidence Dilution and the Best Number of Chunks
- Chapter 4 · Multi-Agent Pipelines: Error Cascades and the Verifier-Gated Absorbing Markov Chain
- Chapter 5 · Verifier Independence and the Cost of Retries
- Chapter 6 · The Statistics of Evaluation Itself: Sample Size, Judge Bias and the Winner's Curse
- Chapter 7 · The Routing of the *Qilue* and Han Fei's *Canyan*: Two Ancient Methods of Retrieval and Verification
- Chapter 8 · A Five-Step Check and a Pre-Registered Offline Evaluation