5D Library › Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents · 中文
Chapter 5. Retrieval Augmentation: One-Book Recall and Cross-Book Synthesis
Retrieval-augmented generation
Retrieval-augmented generation (RAG) joins a retriever to a generative model: first find the passages in a document collection most relevant to a question, then let the model answer from them (Lewis et al. 2020). For questions about a single fact the method works well. The author’s design goes further: let the model draw on several of the library’s 82 volumes at once and synthesise across them. This chapter simulates both kinds of question.
A simulated library
The program takes 82 volumes (the whole library) of 40 passages each, embedded in a 24-dimensional space: each volume has a topic direction, and its passages mix that direction with directions of their own. A one-book question points at one passage plus noise of size ; a cross-book question points at the mean of one passage from each of three volumes, plus noise. Passages are ranked by cosine similarity.
Calculation
# Chapter 5. p09: Recall at k in an 82-volume library, one book or three.
# Each of 82 volumes has 40 passages. In a 24-dimensional embedding, a
# volume has a topic vector t, and its passages are 0.8 t + 0.6 u (u its
# own direction), normalised. A single-book question points at one
# passage plus noise of size sigma; a cross-book question points at the
# mean of three passages from three different volumes plus noise.
# Passages are ranked by cosine similarity. Over 150 questions of each
# kind (fixed seed) the program reports recall at k: the share of
# questions whose needed passage (or all three) are in the top k.
import math
import random
BOOKS, PER, DIM = 82, 40, 24
def unit(v):
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
def corpus(rng):
out = []
for b in range(BOOKS):
t = unit([rng.gauss(0, 1) for _ in range(DIM)])
for _ in range(PER):
u = unit([rng.gauss(0, 1) for _ in range(DIM)])
out.append((b, unit([0.8 * x + 0.6 * y for x, y in zip(t, u)])))
return out
def ranks(cor, q):
q = unit(q)
sc = [sum(a * b for a, b in zip(v, q)) for _, v in cor]
return sorted(range(len(cor)), key=lambda i: -sc[i])
def run(rng, cor, sigma, nneed, ks):
hits = {k: 0 for k in ks}
for _ in range(150):
books = rng.sample(range(BOOKS), nneed)
need = [b * PER + rng.randrange(PER) for b in books]
q = [sum(cor[i][1][d] for i in need) / nneed +
sigma * rng.gauss(0, 1) / math.sqrt(DIM) for d in range(DIM)]
order = ranks(cor, q)
pos = {i: r for r, i in enumerate(order)}
worst = max(pos[i] for i in need)
for k in ks:
hits[k] += worst < k
return {k: hits[k] / 150 for k in ks}
if __name__ == "__main__":
rng = random.Random(76009)
cor = corpus(rng)
ks = (5, 10, 20, 40)
print(" question sigma " + " ".join("{:>7s}".format(
"k={}".format(k)) for k in ks))
for nneed, name in ((1, "one book"), (3, "three")):
for sigma in (0.5, 1.0):
r = run(rng, cor, sigma, nneed, ks)
print(" {:9s} {:6.1f} ".format(name, sigma) + " ".join(
"{:7.3f}".format(r[k]) for k in ks))
print("\nReading: a question about one book finds its passage in a")
print("short list. A question that needs three books at once is far")
print("harder: all three must outrank thousands of passages, and")
print("plain top-k retrieval misses them even at k = 40.")
# question sigma k=5 k=10 k=20 k=40
# one book 0.5 1.000 1.000 1.000 1.000
# one book 1.0 0.913 0.980 0.987 0.993
# three 0.5 0.000 0.007 0.013 0.060
# three 1.0 0.000 0.000 0.000 0.000
#
# Reading: a question about one book finds its passage in a
# short list. A question that needs three books at once is far
# harder: all three must outrank thousands of passages, and
# plain top-k retrieval misses them even at k = 40.A one-book question almost always finds its passage in a very short list: with noise 0.5 recall in the top 5 is 1.000; with noise 1.0 it is 0.913 in the top 5 and 0.993 in the top 40. Cross-book questions are another matter: requiring all three passages in the top , with noise 0.5 the top 20 succeed only 0.013 of the time and the top 40 only 0.060; with noise 1.0, never. A question vector blending three volumes is not close enough to any of their particular passages.
Quotas and splitting
A natural remedy is diversity: at most one passage per volume, forcing the results to cover more volumes (akin to maximal-marginal-relevance reranking) (Carbonell and Goldstein 1998). Another is to split the question into its three parts first and retrieve for each. The program compares three strategies; when splitting, the model splits correctly with probability and falls back to direct retrieval otherwise.
Calculation
# Chapter 5. p10: Finding three books at once: quotas and decomposition.
# Same library model as p09 (82 volumes x 40 passages, 24 dimensions)
# and cross-book questions needing one passage from each of three
# volumes, with noise sigma = 0.5. Three strategies with k passages:
# PLAIN - top k by cosine to the whole question;
# QUOTA - at most one passage per volume, best volumes first (so k
# distinct volumes);
# SPLIT - the agent splits the question into its three parts and
# retrieves top k/3 for each; it splits correctly with
# probability s (1.0 or 0.7), otherwise it falls back to
# PLAIN.
# Success: all three needed passages retrieved. 150 questions each.
import math
import random
BOOKS, PER, DIM = 82, 40, 24
def unit(v):
n = math.sqrt(sum(x * x for x in v))
return [x / n for x in v]
def corpus(rng):
out = []
for b in range(BOOKS):
t = unit([rng.gauss(0, 1) for _ in range(DIM)])
for _ in range(PER):
u = unit([rng.gauss(0, 1) for _ in range(DIM)])
out.append(unit([0.8 * x + 0.6 * y for x, y in zip(t, u)]))
return out
def order(cor, q):
q = unit(q)
sc = [sum(a * b for a, b in zip(v, q)) for v in cor]
return sorted(range(len(cor)), key=lambda i: -sc[i])
def noisy(rng, v, sigma=0.5):
return [x + sigma * rng.gauss(0, 1) / math.sqrt(DIM) for x in v]
def trial(rng, cor, k, strategy, s=1.0):
books = rng.sample(range(BOOKS), 3)
need = [b * PER + rng.randrange(PER) for b in books]
whole = noisy(rng, [sum(cor[i][d] for i in need) / 3
for d in range(DIM)])
if strategy == "SPLIT" and rng.random() < s:
got = set()
for i in need:
got |= set(order(cor, noisy(rng, cor[i]))[:k // 3])
elif strategy == "QUOTA":
got, seen = set(), set()
for i in order(cor, whole):
if i // PER not in seen:
seen.add(i // PER)
got.add(i)
if len(got) == k:
break
else:
got = set(order(cor, whole)[:k])
return all(i in got for i in need)
if __name__ == "__main__":
rng = random.Random(76010)
cor = corpus(rng)
print(" strategy k=15 k=30")
for name, s in (("PLAIN", 1.0), ("QUOTA", 1.0), ("SPLIT", 1.0),
("SPLIT", 0.7)):
row = [sum(trial(rng, cor, k, name, s) for _ in range(150)) / 150
for k in (15, 30)]
label = name if s == 1.0 else "{} s={}".format(name, s)
print(" {:12s} {:6.3f} {:7.3f}".format(label, *row))
print("\nReading: diversity quotas do not rescue a blended question,")
print("because the best passage of a needed volume is not the one")
print("the question points at. Splitting the question into its parts")
print("does: synthesis across books is a planning problem first and")
print("a retrieval problem second.")
# strategy k=15 k=30
# PLAIN 0.013 0.073
# QUOTA 0.007 0.000
# SPLIT 1.000 1.000
# SPLIT s=0.7 0.687 0.767
#
# Reading: diversity quotas do not rescue a blended question,
# because the best passage of a needed volume is not the one
# the question points at. Splitting the question into its parts
# does: synthesis across books is a planning problem first and
# a retrieval problem second.Diversity quotas barely help (0.007 in the top 15): the passage of each volume most similar to a blended question is not the one the question needs. Splitting the question changes everything: split correctly, success is 1.000; with only seven splits in ten correct, success lies between 0.687 and 0.767, set by the accuracy of the split. Synthesis across books is first a planning problem, deciding what to look for in which volumes, and only then a retrieval problem. The library graph of Chapter 6 is the map such planning can rely on.
Hypothesis · not established
The author’s interpretation (not yet tested): in the author’s question-answering system over this library, having the model split cross-book questions into per-volume sub-questions with the help of the library graph, then retrieve for each, clearly raises the accuracy of cross-book synthesis. What would show this wrong: if, on a pre-registered evaluation set, split retrieval and direct retrieval did not differ clearly in accuracy on cross-book questions, cross-book questions in real embeddings would not be as hard as in this chapter’s model.
Summary
One-book questions find their passage in a very short list; questions needing one passage from each of three volumes succeed only 0.060 of the time with direct retrieval even in the top 40. One-passage-per-volume quotas do not help; splitting the question first makes success depend on the accuracy of the split.
References
Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents: Multidimensional State Vectors, Personalized Prompt Conditioning and Cross-Corpus Synthesis [五维灵商量化架构、先验条件化生成与检索增强认知体] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).