5D Library › 五维灵商量化架构、先验条件化生成与检索增强认知体 · English
第五章 检索增强:单书召回与跨书综合
检索增强生成
检索增强生成(RAG)把一个检索器与一个生成模型接在一起:先从文档库中找出与问题最相关的段落,再让模型据此作答 (Lewis et al. 2020)。对单一事实的问题,这种方法很有效。作者的设想更进一步:让模型从全库 82 卷书中同时取出几卷的内容,进行跨书综合。本章模拟这两类问题。
一个书库的模拟
程序设 82 卷书(本书库的全部卷数),每卷 40 段,嵌入在 24 维空间中:每卷有一个主题方向,其段落由主题方向与各自的方向混合而成。单书问题指向某一段,加上大小为 的噪声;跨书问题指向三卷书中各一段的平均,加上噪声。段落按余弦相似度排序。
推导
# Chapter 5. p09: Recall at k in an 82-volume library, one book or three.
# Each of 82 volumes has 40 passages. In a 24-dimensional embedding, a
# volume has a topic vector t, and its passages are 0.8 t + 0.6 u (u its
# own direction), normalised. A single-book question points at one
# passage plus noise of size sigma; a cross-book question points at the
# mean of three passages from three different volumes plus noise.
# Passages are ranked by cosine similarity. Over 150 questions of each
# kind (fixed seed) the program reports recall at k: the share of
# questions whose needed passage (or all three) are in the top k.
import math
import random
BOOKS, PER, DIM = 82, 40, 24
def unit(v):
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
def corpus(rng):
out = []
for b in range(BOOKS):
t = unit([rng.gauss(0, 1) for _ in range(DIM)])
for _ in range(PER):
u = unit([rng.gauss(0, 1) for _ in range(DIM)])
out.append((b, unit([0.8 * x + 0.6 * y for x, y in zip(t, u)])))
return out
def ranks(cor, q):
q = unit(q)
sc = [sum(a * b for a, b in zip(v, q)) for _, v in cor]
return sorted(range(len(cor)), key=lambda i: -sc[i])
def run(rng, cor, sigma, nneed, ks):
hits = {k: 0 for k in ks}
for _ in range(150):
books = rng.sample(range(BOOKS), nneed)
need = [b * PER + rng.randrange(PER) for b in books]
q = [sum(cor[i][1][d] for i in need) / nneed +
sigma * rng.gauss(0, 1) / math.sqrt(DIM) for d in range(DIM)]
order = ranks(cor, q)
pos = {i: r for r, i in enumerate(order)}
worst = max(pos[i] for i in need)
for k in ks:
hits[k] += worst < k
return {k: hits[k] / 150 for k in ks}
if __name__ == "__main__":
rng = random.Random(76009)
cor = corpus(rng)
ks = (5, 10, 20, 40)
print(" question sigma " + " ".join("{:>7s}".format(
"k={}".format(k)) for k in ks))
for nneed, name in ((1, "one book"), (3, "three")):
for sigma in (0.5, 1.0):
r = run(rng, cor, sigma, nneed, ks)
print(" {:9s} {:6.1f} ".format(name, sigma) + " ".join(
"{:7.3f}".format(r[k]) for k in ks))
print("\nReading: a question about one book finds its passage in a")
print("short list. A question that needs three books at once is far")
print("harder: all three must outrank thousands of passages, and")
print("plain top-k retrieval misses them even at k = 40.")
# question sigma k=5 k=10 k=20 k=40
# one book 0.5 1.000 1.000 1.000 1.000
# one book 1.0 0.913 0.980 0.987 0.993
# three 0.5 0.000 0.007 0.013 0.060
# three 1.0 0.000 0.000 0.000 0.000
#
# Reading: a question about one book finds its passage in a
# short list. A question that needs three books at once is far
# harder: all three must outrank thousands of passages, and
# plain top-k retrieval misses them even at k = 40.单书问题几乎总能在很短的列表中找到所需段落:噪声为 0.5 时前 5 段的召回率为 1.000,噪声为 1.0 时前 5 段为 0.913、前 40 段为 0.993。跨书问题则完全不同:要求三段同时进入前 段,噪声为 0.5 时前 20 段只有 0.013,前 40 段也只有 0.060;噪声为 1.0 时为 0。一个混合了三卷书内容的问题向量,离每一卷的具体段落都不够近。
配额与拆分
一个自然的补救是多样化:每卷书最多取一段,强迫结果覆盖更多的卷(类似最大边际相关的重排序)(Carbonell and Goldstein 1998)。另一个办法是先把问题拆成三个部分,分别检索。程序比较三种策略;拆分时,模型以概率 拆对,拆错时退回直接检索。
推导
# Chapter 5. p10: Finding three books at once: quotas and decomposition.
# Same library model as p09 (82 volumes x 40 passages, 24 dimensions)
# and cross-book questions needing one passage from each of three
# volumes, with noise sigma = 0.5. Three strategies with k passages:
# PLAIN - top k by cosine to the whole question;
# QUOTA - at most one passage per volume, best volumes first (so k
# distinct volumes);
# SPLIT - the agent splits the question into its three parts and
# retrieves top k/3 for each; it splits correctly with
# probability s (1.0 or 0.7), otherwise it falls back to
# PLAIN.
# Success: all three needed passages retrieved. 150 questions each.
import math
import random
BOOKS, PER, DIM = 82, 40, 24
def unit(v):
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
def corpus(rng):
out = []
for b in range(BOOKS):
t = unit([rng.gauss(0, 1) for _ in range(DIM)])
for _ in range(PER):
u = unit([rng.gauss(0, 1) for _ in range(DIM)])
out.append(unit([0.8 * x + 0.6 * y for x, y in zip(t, u)]))
return out
def order(cor, q):
q = unit(q)
sc = [sum(a * b for a, b in zip(v, q)) for v in cor]
return sorted(range(len(cor)), key=lambda i: -sc[i])
def noisy(rng, v, sigma=0.5):
return [x + sigma * rng.gauss(0, 1) / math.sqrt(DIM) for x in v]
def trial(rng, cor, k, strategy, s=1.0):
books = rng.sample(range(BOOKS), 3)
need = [b * PER + rng.randrange(PER) for b in books]
whole = noisy(rng, [sum(cor[i][d] for i in need) / 3
for d in range(DIM)])
if strategy == "SPLIT" and rng.random() < s:
got = set()
for i in need:
got |= set(order(cor, noisy(rng, cor[i]))[:k // 3])
elif strategy == "QUOTA":
got, seen = set(), set()
for i in order(cor, whole):
if i // PER not in seen:
seen.add(i // PER)
got.add(i)
if len(got) == k:
break
else:
got = set(order(cor, whole)[:k])
return all(i in got for i in need)
if __name__ == "__main__":
rng = random.Random(76010)
cor = corpus(rng)
print(" strategy k=15 k=30")
for name, s in (("PLAIN", 1.0), ("QUOTA", 1.0), ("SPLIT", 1.0),
("SPLIT", 0.7)):
row = [sum(trial(rng, cor, k, name, s) for _ in range(150)) / 150
for k in (15, 30)]
label = name if s == 1.0 else "{} s={}".format(name, s)
print(" {:12s} {:6.3f} {:7.3f}".format(label, *row))
print("\nReading: diversity quotas do not rescue a blended question,")
print("because the best passage of a needed volume is not the one")
print("the question points at. Splitting the question into its parts")
print("does: synthesis across books is a planning problem first and")
print("a retrieval problem second.")
# strategy k=15 k=30
# PLAIN 0.013 0.073
# QUOTA 0.007 0.000
# SPLIT 1.000 1.000
# SPLIT s=0.7 0.687 0.767
#
# Reading: diversity quotas do not rescue a blended question,
# because the best passage of a needed volume is not the one
# the question points at. Splitting the question into its parts
# does: synthesis across books is a planning problem first and
# a retrieval problem second.多样化配额几乎没有帮助(前 15 段 0.007):每卷书中与混合问题最相似的段落,并不是问题真正需要的那一段。拆分问题则彻底改变了结果:拆对时成功率为 1.000;只有七成能拆对时,成功率也在 0.687 至 0.767 之间,由拆分的准确率决定。跨书综合首先是一个规划问题,即先想清楚要从哪几卷书中找什么,其次才是检索问题。第六章的书库关联图正是这种规划可以依靠的地图。
假说 · 尚未验证
作者的诠释(尚未验证): 在作者基于本书库的问答系统中,先由模型根据书库关联图把跨书问题拆成按卷的子问题、再分别检索,可以显著提高跨书综合的正确率。什么会证明这一判断错:若在预先登记的评测集上,拆分检索与直接检索在跨书问题上的正确率没有显著差别,说明真实嵌入中的跨书问题并不像本章的模型那样难。
小结
单书问题在很短的检索列表中就能找到所需段落;需要同时找到三卷书中各一段的问题,直接检索在前 40 段中也只有 0.060 的成功率。每卷限一段的多样化配额无济于事;先拆分问题再检索,成功率由拆分的准确率决定。
参考文献
Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents: Multidimensional State Vectors, Personalized Prompt Conditioning and Cross-Corpus Synthesis [五维灵商量化架构、先验条件化生成与检索增强认知体] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).