5D Library › 五维灵商量化架构、先验条件化生成与检索增强认知体 · English

第五章 检索增强:单书召回与跨书综合

检索增强生成

检索增强生成(RAG)把一个检索器与一个生成模型接在一起:先从文档库中找出与问题最相关的段落,再让模型据此作答 (Lewis et al. 2020)。对单一事实的问题,这种方法很有效。作者的设想更进一步:让模型从全库 82 卷书中同时取出几卷的内容,进行跨书综合。本章模拟这两类问题。

一个书库的模拟

程序设 82 卷书(本书库的全部卷数),每卷 40 段,嵌入在 24 维空间中:每卷有一个主题方向,其段落由主题方向与各自的方向混合而成。单书问题指向某一段,加上大小为 σ\sigma 的噪声;跨书问题指向三卷书中各一段的平均,加上噪声。段落按余弦相似度排序。

推导

# Chapter 5. p09: Recall at k in an 82-volume library, one book or three.
# Each of 82 volumes has 40 passages. In a 24-dimensional embedding, a
# volume has a topic vector t, and its passages are 0.8 t + 0.6 u (u its
# own direction), normalised. A single-book question points at one
# passage plus noise of size sigma; a cross-book question points at the
# mean of three passages from three different volumes plus noise.
# Passages are ranked by cosine similarity. Over 150 questions of each
# kind (fixed seed) the program reports recall at k: the share of
# questions whose needed passage (or all three) are in the top k.
import math
import random

BOOKS, PER, DIM = 82, 40, 24

def unit(v):
    n = math.sqrt(sum(x * x for x in v))
    return [x / n for x in v]

def corpus(rng):
    out = []
    for b in range(BOOKS):
        t = unit([rng.gauss(0, 1) for _ in range(DIM)])
        for _ in range(PER):
            u = unit([rng.gauss(0, 1) for _ in range(DIM)])
            out.append((b, unit([0.8 * x + 0.6 * y for x, y in zip(t, u)])))
    return out

def ranks(cor, q):
    q = unit(q)
    sc = [sum(a * b for a, b in zip(v, q)) for _, v in cor]
    return sorted(range(len(cor)), key=lambda i: -sc[i])

def run(rng, cor, sigma, nneed, ks):
    hits = {k: 0 for k in ks}
    for _ in range(150):
        books = rng.sample(range(BOOKS), nneed)
        need = [b * PER + rng.randrange(PER) for b in books]
        q = [sum(cor[i][1][d] for i in need) / nneed +
             sigma * rng.gauss(0, 1) / math.sqrt(DIM) for d in range(DIM)]
        order = ranks(cor, q)
        pos = {i: r for r, i in enumerate(order)}
        worst = max(pos[i] for i in need)
        for k in ks:
            hits[k] += worst < k
    return {k: hits[k] / 150 for k in ks}

if __name__ == "__main__":
    rng = random.Random(76009)
    cor = corpus(rng)
    ks = (5, 10, 20, 40)
    print("   question   sigma  " + " ".join("{:>7s}".format(
        "k={}".format(k)) for k in ks))
    for nneed, name in ((1, "one book"), (3, "three")):
        for sigma in (0.5, 1.0):
            r = run(rng, cor, sigma, nneed, ks)
            print("   {:9s} {:6.1f}  ".format(name, sigma) + " ".join(
                "{:7.3f}".format(r[k]) for k in ks))
    print("\nReading: a question about one book finds its passage in a")
    print("short list. A question that needs three books at once is far")
    print("harder: all three must outrank thousands of passages, and")
    print("plain top-k retrieval misses them even at k = 40.")
#    question   sigma      k=5    k=10    k=20    k=40
#    one book     0.5    1.000   1.000   1.000   1.000
#    one book     1.0    0.913   0.980   0.987   0.993
#    three        0.5    0.000   0.007   0.013   0.060
#    three        1.0    0.000   0.000   0.000   0.000
#
# Reading: a question about one book finds its passage in a
# short list. A question that needs three books at once is far
# harder: all three must outrank thousands of passages, and
# plain top-k retrieval misses them even at k = 40.
单书问题与三书综合问题的召回率随检索段数变化。

单书问题几乎总能在很短的列表中找到所需段落:噪声为 0.5 时前 5 段的召回率为 1.000,噪声为 1.0 时前 5 段为 0.913、前 40 段为 0.993。跨书问题则完全不同:要求三段同时进入前 kk 段,噪声为 0.5 时前 20 段只有 0.013,前 40 段也只有 0.060;噪声为 1.0 时为 0。一个混合了三卷书内容的问题向量,离每一卷的具体段落都不够近。

配额与拆分

一个自然的补救是多样化:每卷书最多取一段,强迫结果覆盖更多的卷(类似最大边际相关的重排序)(Carbonell and Goldstein 1998)。另一个办法是先把问题拆成三个部分,分别检索。程序比较三种策略;拆分时,模型以概率 ss 拆对,拆错时退回直接检索。

推导

# Chapter 5. p10: Finding three books at once: quotas and decomposition.
# Same library model as p09 (82 volumes x 40 passages, 24 dimensions)
# and cross-book questions needing one passage from each of three
# volumes, with noise sigma = 0.5. Three strategies with k passages:
#   PLAIN  - top k by cosine to the whole question;
#   QUOTA  - at most one passage per volume, best volumes first (so k
#            distinct volumes);
#   SPLIT  - the agent splits the question into its three parts and
#            retrieves top k/3 for each; it splits correctly with
#            probability s (1.0 or 0.7), otherwise it falls back to
#            PLAIN.
# Success: all three needed passages retrieved. 150 questions each.
import math
import random

BOOKS, PER, DIM = 82, 40, 24

def unit(v):
    n = math.sqrt(sum(x * x for x in v))
    return [x / n for x in v]

def corpus(rng):
    out = []
    for b in range(BOOKS):
        t = unit([rng.gauss(0, 1) for _ in range(DIM)])
        for _ in range(PER):
            u = unit([rng.gauss(0, 1) for _ in range(DIM)])
            out.append(unit([0.8 * x + 0.6 * y for x, y in zip(t, u)]))
    return out

def order(cor, q):
    q = unit(q)
    sc = [sum(a * b for a, b in zip(v, q)) for v in cor]
    return sorted(range(len(cor)), key=lambda i: -sc[i])

def noisy(rng, v, sigma=0.5):
    return [x + sigma * rng.gauss(0, 1) / math.sqrt(DIM) for x in v]

def trial(rng, cor, k, strategy, s=1.0):
    books = rng.sample(range(BOOKS), 3)
    need = [b * PER + rng.randrange(PER) for b in books]
    whole = noisy(rng, [sum(cor[i][d] for i in need) / 3
                        for d in range(DIM)])
    if strategy == "SPLIT" and rng.random() < s:
        got = set()
        for i in need:
            got |= set(order(cor, noisy(rng, cor[i]))[:k // 3])
    elif strategy == "QUOTA":
        got, seen = set(), set()
        for i in order(cor, whole):
            if i // PER not in seen:
                seen.add(i // PER)
                got.add(i)
                if len(got) == k:
                    break
    else:
        got = set(order(cor, whole)[:k])
    return all(i in got for i in need)

if __name__ == "__main__":
    rng = random.Random(76010)
    cor = corpus(rng)
    print("   strategy      k=15    k=30")
    for name, s in (("PLAIN", 1.0), ("QUOTA", 1.0), ("SPLIT", 1.0),
                    ("SPLIT", 0.7)):
        row = [sum(trial(rng, cor, k, name, s) for _ in range(150)) / 150
               for k in (15, 30)]
        label = name if s == 1.0 else "{} s={}".format(name, s)
        print("   {:12s} {:6.3f} {:7.3f}".format(label, *row))
    print("\nReading: diversity quotas do not rescue a blended question,")
    print("because the best passage of a needed volume is not the one")
    print("the question points at. Splitting the question into its parts")
    print("does: synthesis across books is a planning problem first and")
    print("a retrieval problem second.")
#    strategy      k=15    k=30
#    PLAIN         0.013   0.073
#    QUOTA         0.007   0.000
#    SPLIT         1.000   1.000
#    SPLIT s=0.7   0.687   0.767
#
# Reading: diversity quotas do not rescue a blended question,
# because the best passage of a needed volume is not the one
# the question points at. Splitting the question into its parts
# does: synthesis across books is a planning problem first and
# a retrieval problem second.

多样化配额几乎没有帮助(前 15 段 0.007):每卷书中与混合问题最相似的段落,并不是问题真正需要的那一段。拆分问题则彻底改变了结果:拆对时成功率为 1.000;只有七成能拆对时,成功率也在 0.687 至 0.767 之间,由拆分的准确率决定。跨书综合首先是一个规划问题,即先想清楚要从哪几卷书中找什么,其次才是检索问题。第六章的书库关联图正是这种规划可以依靠的地图。

假说 · 尚未验证

作者的诠释(尚未验证): 在作者基于本书库的问答系统中,先由模型根据书库关联图把跨书问题拆成按卷的子问题、再分别检索,可以显著提高跨书综合的正确率。什么会证明这一判断错:若在预先登记的评测集上,拆分检索与直接检索在跨书问题上的正确率没有显著差别,说明真实嵌入中的跨书问题并不像本章的模型那样难。

小结

单书问题在很短的检索列表中就能找到所需段落;需要同时找到三卷书中各一段的问题,直接检索在前 40 段中也只有 0.060 的成功率。每卷限一段的多样化配额无济于事;先拆分问题再检索,成功率由拆分的准确率决定。

参考文献

Carbonell, Jaime, and Jade Goldstein. 1998. “The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries.” Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 335–36.
Lewis, Patrick, Ethan Perez, Aleksandra Piktus, et al. 2020. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” Advances in Neural Information Processing Systems 33, 9459–74.

Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents: Multidimensional State Vectors, Personalized Prompt Conditioning and Cross-Corpus Synthesis [五维灵商量化架构、先验条件化生成与检索增强认知体] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).