关于「高维几何:测度集中、有效维度与枢纽块」,《检索增强与多智能体的可靠性》给出的定量结论是什么?

结论:第一章用测度集中、有效维度与枢纽现象说明:语料越大,第一名错误率越高(256 维时从 0.0730 升到 0.5030);名义 768 维、方差按幂律衰减时,有效维度只有 31.7,十万块中排错第一名的概率 0.9875;一百维时 1% 的枢纽块闯入 77.7% 的无关问题的前十名。

数据口径:全部数据为程序生成的合成数据,数字说明机制与量级,不是对任何产品的测量。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0730, 0.5030, 256, 768

A. cosine of random unit vectors
d sd (simulated)1/sqrt(d) largest |cos| of 2,000
160.2524 0.2500 0.8060
640.1243 0.1250 0.4137
2560.0623 0.0625 0.1969
10240.0320 0.0312 0.1029
B. P(top-1 wrong), relevant cosine 0.30 (+ noise of same sd)
d M1000 M = 10000 M = 100000 M = 1000000
640.7890 0.9270 0.9710 0.9850
2560.0730 0.1740 0.3280 0.5030
7680.0000 0.0000 0.0000 0.0010
d64: sqrt(2 ln M / d) at M = 1e6 is 0.657
d768: sqrt(2 ln M / d) at M = 1e6 is 0.190
Reading: in this isotropic model more dimensions help - noise
cosines shrink like 1/sqrt(d) - but every tenfold growth of the
corpus raises the best distractor by a fixed step, so top-1
errors climb with M at any d. The next program shows
why real embeddings behave as if d were far smaller.

核心公式

\[d_{\mathrm{eff}} = \frac{\big(\sum_i \lambda_i\big)^2}{\sum_i \lambda_i^2} ,\]
查看完整验算程序
# Chapter 1. p01: Concentration of measure and the corpus-size penalty.
# A. Cosines between independent random unit vectors in d dimensions
#    have mean 0 and standard deviation 1/sqrt(d) (2,000 pairs each).
# B. Retrieval as an extreme-value race. The relevant chunk has cosine
#    s = 0.30 with the query plus noise; each of M irrelevant chunks has a
#    cosine that is pure noise of sd 1/sqrt(d) (isotropic model). Top-1 is
#    wrong when the largest of M noise cosines beats the relevant one.
#    The largest of M grows like sqrt(2 ln M / d). 1,000 queries per cell.
import math
import random

def unit(rng, d):
    v = [rng.gauss(0, 1) for _ in range(d)]
    n = math.sqrt(sum(x * x for x in v))
    return [x / n for x in v]

def erfinv(y):
    # Newton on erf(x) = y, from a good start
    x = 0.0 if abs(y) < 0.7 else math.copysign(
        math.sqrt(-math.log((1 - abs(y)) / 2)), y)
    for _ in range(60):
        err = math.erf(x) - y
        x -= err / (2 / math.sqrt(math.pi) * math.exp(-x * x))
    return x

if __name__ == "__main__":
    rng = random.Random(61001)
    print("A. cosine of random unit vectors")
    print("      d    sd (simulated)   1/sqrt(d)   largest |cos| of 2,000")
    for d in (16, 64, 256, 1024):
        cs = []
        for _ in range(2000):
            a, b = unit(rng, d), unit(rng, d)
            cs.append(sum(x * y for x, y in zip(a, b)))
        sd = math.sqrt(sum(c * c for c in cs) / len(cs))
        print("   {:4d}      {:.4f}          {:.4f}       {:.4f}".format(
            d, sd, 1 / math.sqrt(d), max(abs(c) for c in cs)))
    print("\nB. P(top-1 wrong), relevant cosine 0.30 (+ noise of same sd)")
    sizes = (1000, 10000, 100000, 1000000)
    print("      d   " + "".join("   M = {:<8d}".format(m) for m in sizes))
    for d in (64, 256, 768):
        sd = 1 / math.sqrt(d)
        cells = []
        for m in sizes:
            wrong = 0
            for _ in range(1000):
                rel = 0.30 + rng.gauss(0, sd)
                # max of m iid N(0, sd^2) by inverse CDF of the max
                u = rng.random() ** (1 / m)
                z = math.sqrt(2) * erfinv(2 * u - 1)
                wrong += z * sd > rel
            cells.append("   {:9.4f}  ".format(wrong / 1000))
        print("   {:4d}".format(d) + "".join(cells))
    for d in (64, 768):
        print("   d = {:3d}: sqrt(2 ln M / d) at M = 1e6 is {:.3f}".format(
            d, math.sqrt(2 * math.log(1e6) / d)))
    print("\nReading: in this isotropic model more dimensions help - noise")
    print("cosines shrink like 1/sqrt(d) - but every tenfold growth of the")
    print("corpus raises the best distractor by a fixed step, so top-1")
    print("errors climb with M at any d. The next program shows")
    print("why real embeddings behave as if d were far smaller.")

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《检索增强生成几何、倒数秩融合与多智能体误差级联》第 1 章「高维几何:测度集中、有效维度与枢纽块」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090170

DOI: 10.5281/zenodo.24090170 · 阅读本章所在专著 · 5DL-B-061 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026061,
  author    = {Fan, Yafu},
  title     = {Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.24090170},
  url       = {https://doi.org/10.5281/zenodo.24090170}
}

本书其他章节结论