关于「高维几何:测度集中、有效维度与枢纽块」,《检索增强与多智能体的可靠性》给出的定量结论是什么?
结论:第一章用测度集中、有效维度与枢纽现象说明:语料越大,第一名错误率越高(256 维时从 0.0730 升到 0.5030);名义 768 维、方差按幂律衰减时,有效维度只有 31.7,十万块中排错第一名的概率 0.9875;一百维时 1% 的枢纽块闯入 77.7% 的无关问题的前十名。
数据口径:全部数据为程序生成的合成数据,数字说明机制与量级,不是对任何产品的测量。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0730, 0.5030, 256, 768
| A. cosine of random unit vectors | |
| d sd (simulated) | 1/sqrt(d) largest |cos| of 2,000 |
| 16 | 0.2524 0.2500 0.8060 |
| 64 | 0.1243 0.1250 0.4137 |
| 256 | 0.0623 0.0625 0.1969 |
| 1024 | 0.0320 0.0312 0.1029 |
| B. P(top-1 wrong), relevant cosine 0.30 (+ noise of same sd) | |
| d M | 1000 M = 10000 M = 100000 M = 1000000 |
| 64 | 0.7890 0.9270 0.9710 0.9850 |
| 256 | 0.0730 0.1740 0.3280 0.5030 |
| 768 | 0.0000 0.0000 0.0000 0.0010 |
| d | 64: sqrt(2 ln M / d) at M = 1e6 is 0.657 |
| d | 768: sqrt(2 ln M / d) at M = 1e6 is 0.190 |
| Reading: in this isotropic model more dimensions help - noise | |
| cosines shrink like 1/sqrt(d) - but every tenfold growth of the | |
| corpus raises the best distractor by a fixed step, so top-1 | |
| errors climb with M at any d. The next program shows | |
| why real embeddings behave as if d were far smaller. |
核心公式
\[d_{\mathrm{eff}} = \frac{\big(\sum_i \lambda_i\big)^2}{\sum_i \lambda_i^2} ,\]
查看完整验算程序
# Chapter 1. p01: Concentration of measure and the corpus-size penalty.
# A. Cosines between independent random unit vectors in d dimensions
# have mean 0 and standard deviation 1/sqrt(d) (2,000 pairs each).
# B. Retrieval as an extreme-value race. The relevant chunk has cosine
# s = 0.30 with the query plus noise; each of M irrelevant chunks has a
# cosine that is pure noise of sd 1/sqrt(d) (isotropic model). Top-1 is
# wrong when the largest of M noise cosines beats the relevant one.
# The largest of M grows like sqrt(2 ln M / d). 1,000 queries per cell.
import math
import random
def unit(rng, d):
v = [rng.gauss(0, 1) for _ in range(d)]
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
def erfinv(y):
# Newton on erf(x) = y, from a good start
x = 0.0 if abs(y) < 0.7 else math.copysign(
math.sqrt(-math.log((1 - abs(y)) / 2)), y)
for _ in range(60):
err = math.erf(x) - y
x -= err / (2 / math.sqrt(math.pi) * math.exp(-x * x))
return x
if __name__ == "__main__":
rng = random.Random(61001)
print("A. cosine of random unit vectors")
print(" d sd (simulated) 1/sqrt(d) largest |cos| of 2,000")
for d in (16, 64, 256, 1024):
cs = []
for _ in range(2000):
a, b = unit(rng, d), unit(rng, d)
cs.append(sum(x * y for x, y in zip(a, b)))
sd = math.sqrt(sum(c * c for c in cs) / len(cs))
print(" {:4d} {:.4f} {:.4f} {:.4f}".format(
d, sd, 1 / math.sqrt(d), max(abs(c) for c in cs)))
print("\nB. P(top-1 wrong), relevant cosine 0.30 (+ noise of same sd)")
sizes = (1000, 10000, 100000, 1000000)
print(" d " + "".join(" M = {:<8d}".format(m) for m in sizes))
for d in (64, 256, 768):
sd = 1 / math.sqrt(d)
cells = []
for m in sizes:
wrong = 0
for _ in range(1000):
rel = 0.30 + rng.gauss(0, sd)
# max of m iid N(0, sd^2) by inverse CDF of the max
u = rng.random() ** (1 / m)
z = math.sqrt(2) * erfinv(2 * u - 1)
wrong += z * sd > rel
cells.append(" {:9.4f} ".format(wrong / 1000))
print(" {:4d}".format(d) + "".join(cells))
for d in (64, 768):
print(" d = {:3d}: sqrt(2 ln M / d) at M = 1e6 is {:.3f}".format(
d, math.sqrt(2 * math.log(1e6) / d)))
print("\nReading: in this isotropic model more dimensions help - noise")
print("cosines shrink like 1/sqrt(d) - but every tenfold growth of the")
print("corpus raises the best distractor by a fixed step, so top-1")
print("errors climb with M at any d. The next program shows")
print("why real embeddings behave as if d were far smaller.")唯一数据源与引用
范雅福(Fan, Y.). (2026). 《检索增强生成几何、倒数秩融合与多智能体误差级联》第 1 章「高维几何:测度集中、有效维度与枢纽块」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090170
DOI: 10.5281/zenodo.24090170 · 阅读本章所在专著 · 5DL-B-061 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026061,
author = {Fan, Yafu},
title = {Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.24090170},
url = {https://doi.org/10.5281/zenodo.24090170}
}