5D Library › 五维灵商量化架构、先验条件化生成与检索增强认知体 · English
第三章 先验条件化:一份档案能让回答少多少不确定
把问题写成条件熵
作者的设计是:每位用户有一份约 8 KB 的
destiny_profile.json,向量
记录其五维状态与相关信息;模型在回答问题
时,把
与检索到的段落
一起放进提示,生成
。提纲认为这样可以降低回答的条件熵
。
信息论给出的答案很简单,也很严格 (Cover and Thomas 2006):加入 后条件熵降低的量是条件互信息
它不超过 ,也不超过 。也就是说,档案能带来的好处有两个上限:回答本身有多少不确定,以及回答真正依赖档案的程度。文件有多大并不在其中。
一个可以精确计算的例子
程序设 16 类问题、8 类建议、档案为五个维度各三档(243 种状态)。在 ONE 情形中,每类问题只与一个维度有关,合适的建议由该维度决定,以 0.7 的概率给出合适的建议;TWO 情形与两个维度有关;NONE 情形中建议与档案无关。
推导
# Chapter 3. p05: How much can a profile tell the model? Exact entropies.
# X: 16 kinds of question, equally likely. Z: a profile of five
# dimensions with three levels each (243 states, uniform), as stored in
# destiny_profile.json. Y: one of 8 kinds of advice. For question x
# the relevant dimension is k = x mod 5 (case ONE) or k and k + 1 (case
# TWO); the fitting advice is y* = (x + 3 z_k) mod 8 (case TWO adds
# 5 z_k+1), given with probability 0.7, any of the 8 otherwise. Case
# NONE: the advice ignores Z (y* = x mod 8). Computed exactly:
# H(Y|X), H(Y|X,Z) and I(Y;Z|X) = H(Y|X) - H(Y|X,Z), in bits,
# set against H(Z) = log2 243 and the 65,536 bits of an 8 KB file.
import itertools
import math
def h(p):
return -sum(x * math.log2(x) for x in p if x > 0)
def ystar(case, x, z):
k = x % 5
if case == "ONE":
return (x + 3 * z[k]) % 8
if case == "TWO":
return (x + 3 * z[k] + 5 * z[(k + 1) % 5]) % 8
return x % 8
def entropies(case, keep=0.7):
zs = list(itertools.product(range(3), repeat=5))
hyx = hyxz = 0.0
for x in range(16):
py = [0.0] * 8
for z in zs:
ys = ystar(case, x, z)
for y in range(8):
p = (1 - keep) / 8 + (keep if y == ys else 0.0)
py[y] += p / len(zs)
hyx += h(py) / 16
hyxz += h([(1 - keep) / 8 + (keep if y == 0 else 0)
for y in range(8)]) / 16
gain = hyx - hyxz
return hyx, hyxz, gain if abs(gain) > 1e-12 else 0.0
if __name__ == "__main__":
print(" H(Z) = log2 243 = {:.4f} bits; an 8 KB file: {:,d} bits"
.format(math.log2(243), 8 * 1024 * 8))
print(" case H(Y|X) H(Y|X,Z) I(Y;Z|X)")
for case in ("NONE", "ONE", "TWO"):
a, b, c = entropies(case)
print(" {:5s} {:8.4f} {:10.4f} {:9.4f}".format(case, a, b, c))
print(" upper bound on any gain: H(Y) = log2 8 = 3 bits")
print("\nReading: a profile can lower the uncertainty of the advice")
print("by at most what the advice depends on: here about one bit,")
print("against 7.9 bits of profile and 65,536 bits of file. If the")
print("advice ignores the profile, the gain is exactly zero.")
print("Conditioning helps only through a real dependence, which")
print("must be shown, not assumed.")
# H(Z) = log2 243 = 7.9248 bits; an 8 KB file: 65,536 bits
# case H(Y|X) H(Y|X,Z) I(Y;Z|X)
# NONE 1.5674 1.5674 0.0000
# ONE 2.4194 1.5674 0.8519
# TWO 2.6781 1.5674 1.1107
# upper bound on any gain: H(Y) = log2 8 = 3 bits
#
# Reading: a profile can lower the uncertainty of the advice
# by at most what the advice depends on: here about one bit,
# against 7.9 bits of profile and 65,536 bits of file. If the
# advice ignores the profile, the gain is exactly zero.
# Conditioning helps only through a real dependence, which
# must be shown, not assumed.243 种档案状态的熵是 7.9248 比特,一个 8 KB 的文件可以装 65,536 比特;但在 ONE 情形中,档案只能把回答的条件熵从 2.4194 降到 1.5674,即 0.8519 比特;TWO 情形 1.1107 比特。在 NONE 情形中,档案带来的降低恰好为 0。任何档案的收益都不能超过建议本身的 3 比特。
这一结论的含义是:档案的价值不在于它写了多少,而在于回答是否真的依赖其中的内容。若回答对档案的依赖没有被证明,在提示中加入档案只是让回答「看起来」更个人化。
档案出错时
档案的输入可能有错:出生时辰记错、问卷随手作答、资料过时。程序让档案的每个维度以概率 被随机替换,计算收益 。
推导
# Chapter 3. p06: A noisy profile: how fast the gain disappears.
# Case ONE of p05 (advice depends on one dimension of three levels,
# kept with probability 0.7). The stored profile Z' is wrong on each
# dimension with probability e: the level is replaced by one of the
# three at random (a wrong birth hour, a careless answer, an outdated
# questionnaire). The gain I(Y;Z'|X) = H(Y|X) - H(Y|X,Z') is computed
# exactly for e from 0 to 1, with the share of the noiseless gain kept.
import math
def h(p):
return -sum(x * math.log2(x) for x in p if x > 0)
def p_y(x, zk, keep=0.7):
ys = (x + 3 * zk) % 8
return [(1 - keep) / 8 + (keep if y == ys else 0.0) for y in range(8)]
def gain(e):
hyx = hyxz = 0.0
for x in range(16):
prior = [sum(p_y(x, z)[y] for z in range(3)) / 3 for y in range(8)]
hyx += h(prior) / 16
for zp in range(3):
post = [(1 - e) * (z == zp) + e / 3 for z in range(3)]
norm = sum(post)
py = [sum(post[z] * p_y(x, z)[y] for z in range(3)) / norm
for y in range(8)]
hyxz += h(py) / 16 / 3
return hyx - hyxz
if __name__ == "__main__":
g0 = gain(0.0)
print(" error rate e I(Y;Z'|X) bits share of noiseless gain")
for e in (0.0, 0.1, 0.25, 0.5, 0.75, 1.0):
g = max(0.0, gain(e))
print(" {:12.2f} {:16.4f} {:25.4f}".format(e, g, g / g0))
print("\nReading: the gain falls faster than the error rate rises;")
print("a profile wrong one time in four keeps about half of what it")
print("could give. Checking the inputs is worth more than adding")
print("dimensions to an uncertain profile.")
# error rate e I(Y;Z'|X) bits share of noiseless gain
# 0.00 0.8519 1.0000
# 0.10 0.6711 0.7878
# 0.25 0.4557 0.5350
# 0.50 0.2013 0.2363
# 0.75 0.0515 0.0605
# 1.00 0.0000 0.0000
#
# Reading: the gain falls faster than the error rate rises;
# a profile wrong one time in four keeps about half of what it
# could give. Checking the inputs is worth more than adding
# dimensions to an uncertain profile.收益下降得比错误率上升得快:错误率 0.10 时保留 0.7878,0.25 时只剩 0.5350,0.50 时只剩 0.2363。核对档案的输入,比给一份不可靠的档案增加维度更值得。
假说 · 尚未验证
作者的诠释(尚未验证): 在作者的系统中,五维档案确实改变了合适建议的分布(),且出生时间等输入经过核对后,档案的错误率低于 0.10。什么会证明这一判断错:若在第八章的盲比实验中,用户对「本人档案」与「换入陌生人档案」生成的回答没有显著偏好,那么在实际使用中 接近于零。
小结
档案对回答的帮助以条件互信息衡量,它受回答的不确定性与回答对档案的依赖程度两个上限约束,与文件大小无关:本书例子中,7.9248 比特的档案只带来 0.8519 比特的收益;回答不依赖档案时为 0。档案四分之一出错时,收益只剩 0.5350。
参考文献
Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents: Multidimensional State Vectors, Personalized Prompt Conditioning and Cross-Corpus Synthesis [五维灵商量化架构、先验条件化生成与检索增强认知体] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).