5D Library › Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents · 中文
Chapter 3. Prior Conditioning: How Much Uncertainty Can a Profile Remove?
The question as conditional entropy
The author’s design: each user has a
destiny_profile.json of about 8 KB whose vector
records her five-dimensional state and related information; when
answering a question
the model puts
into the prompt together with retrieved passages
and generates from
.
The outline holds that this lowers the conditional entropy
of the answer.
Information theory gives a simple and strict answer (Cover and Thomas 2006): the fall in conditional entropy when is added is the conditional mutual information
which exceeds neither nor . The benefit of a profile has two ceilings, then: how uncertain the answer is to begin with, and how far the answer really depends on the profile. The size of the file is not among them.
An example that can be computed exactly
The program takes 16 kinds of question, 8 kinds of advice and a profile of five dimensions with three levels each (243 states). In case ONE each kind of question concerns one dimension, which determines the fitting advice, given with probability 0.7; case TWO concerns two dimensions; in case NONE the advice ignores the profile.
Calculation
# Chapter 3. p05: How much can a profile tell the model? Exact entropies.
# X: 16 kinds of question, equally likely. Z: a profile of five
# dimensions with three levels each (243 states, uniform), as stored in
# destiny_profile.json. Y: one of 8 kinds of advice. For question x
# the relevant dimension is k = x mod 5 (case ONE) or k and k + 1 (case
# TWO); the fitting advice is y* = (x + 3 z_k) mod 8 (case TWO adds
# 5 z_k+1), given with probability 0.7, any of the 8 otherwise. Case
# NONE: the advice ignores Z (y* = x mod 8). Computed exactly:
# H(Y|X), H(Y|X,Z) and I(Y;Z|X) = H(Y|X) - H(Y|X,Z), in bits,
# set against H(Z) = log2 243 and the 65,536 bits of an 8 KB file.
import itertools
import math
def h(p):
return -sum(x * math.log2(x) for x in p if x > 0)
def ystar(case, x, z):
k = x % 5
if case == "ONE":
return (x + 3 * z[k]) % 8
if case == "TWO":
return (x + 3 * z[k] + 5 * z[(k + 1) % 5]) % 8
return x % 8
def entropies(case, keep=0.7):
zs = list(itertools.product(range(3), repeat=5))
hyx = hyxz = 0.0
for x in range(16):
py = [0.0] * 8
for z in zs:
ys = ystar(case, x, z)
for y in range(8):
p = (1 - keep) / 8 + (keep if y == ys else 0.0)
py[y] += p / len(zs)
hyx += h(py) / 16
hyxz += h([(1 - keep) / 8 + (keep if y == 0 else 0)
for y in range(8)]) / 16
gain = hyx - hyxz
return hyx, hyxz, gain if abs(gain) > 1e-12 else 0.0
if __name__ == "__main__":
print(" H(Z) = log2 243 = {:.4f} bits; an 8 KB file: {:,d} bits"
.format(math.log2(243), 8 * 1024 * 8))
print(" case H(Y|X) H(Y|X,Z) I(Y;Z|X)")
for case in ("NONE", "ONE", "TWO"):
a, b, c = entropies(case)
print(" {:5s} {:8.4f} {:10.4f} {:9.4f}".format(case, a, b, c))
print(" upper bound on any gain: H(Y) = log2 8 = 3 bits")
print("\nReading: a profile can lower the uncertainty of the advice")
print("by at most what the advice depends on: here about one bit,")
print("against 7.9 bits of profile and 65,536 bits of file. If the")
print("advice ignores the profile, the gain is exactly zero.")
print("Conditioning helps only through a real dependence, which")
print("must be shown, not assumed.")
# H(Z) = log2 243 = 7.9248 bits; an 8 KB file: 65,536 bits
# case H(Y|X) H(Y|X,Z) I(Y;Z|X)
# NONE 1.5674 1.5674 0.0000
# ONE 2.4194 1.5674 0.8519
# TWO 2.6781 1.5674 1.1107
# upper bound on any gain: H(Y) = log2 8 = 3 bits
#
# Reading: a profile can lower the uncertainty of the advice
# by at most what the advice depends on: here about one bit,
# against 7.9 bits of profile and 65,536 bits of file. If the
# advice ignores the profile, the gain is exactly zero.
# Conditioning helps only through a real dependence, which
# must be shown, not assumed.The 243 profile states carry 7.9248 bits and an 8 KB file can hold 65,536 bits; yet in case ONE the profile lowers the conditional entropy of the answer only from 2.4194 to 1.5674, that is by 0.8519 bits; in case TWO by 1.1107 bits. In case NONE the profile lowers it by exactly 0. No profile can gain more than the 3 bits of the advice itself.
The implication: a profile’s value lies not in how much it records but in whether answers really depend on what it records. Unless the dependence of answers on the profile has been shown, adding the profile to the prompt only makes answers “look” more personal.
When the profile is wrong
Profile inputs can be wrong: a misremembered birth hour, a careless questionnaire, outdated information. The program replaces each profile dimension at random with probability and computes the gain .
Calculation
# Chapter 3. p06: A noisy profile: how fast the gain disappears.
# Case ONE of p05 (advice depends on one dimension of three levels,
# kept with probability 0.7). The stored profile Z' is wrong on each
# dimension with probability e: the level is replaced by one of the
# three at random (a wrong birth hour, a careless answer, an outdated
# questionnaire). The gain I(Y;Z'|X) = H(Y|X) - H(Y|X,Z') is computed
# exactly for e from 0 to 1, with the share of the noiseless gain kept.
import math
def h(p):
return -sum(x * math.log2(x) for x in p if x > 0)
def p_y(x, zk, keep=0.7):
ys = (x + 3 * zk) % 8
return [(1 - keep) / 8 + (keep if y == ys else 0.0) for y in range(8)]
def gain(e):
hyx = hyxz = 0.0
for x in range(16):
prior = [sum(p_y(x, z)[y] for z in range(3)) / 3 for y in range(8)]
hyx += h(prior) / 16
for zp in range(3):
post = [(1 - e) * (z == zp) + e / 3 for z in range(3)]
norm = sum(post)
py = [sum(post[z] * p_y(x, z)[y] for z in range(3)) / norm
for y in range(8)]
hyxz += h(py) / 16 / 3
return hyx - hyxz
if __name__ == "__main__":
g0 = gain(0.0)
print(" error rate e I(Y;Z'|X) bits share of noiseless gain")
for e in (0.0, 0.1, 0.25, 0.5, 0.75, 1.0):
g = max(0.0, gain(e))
print(" {:12.2f} {:16.4f} {:25.4f}".format(e, g, g / g0))
print("\nReading: the gain falls faster than the error rate rises;")
print("a profile wrong one time in four keeps about half of what it")
print("could give. Checking the inputs is worth more than adding")
print("dimensions to an uncertain profile.")
# error rate e I(Y;Z'|X) bits share of noiseless gain
# 0.00 0.8519 1.0000
# 0.10 0.6711 0.7878
# 0.25 0.4557 0.5350
# 0.50 0.2013 0.2363
# 0.75 0.0515 0.0605
# 1.00 0.0000 0.0000
#
# Reading: the gain falls faster than the error rate rises;
# a profile wrong one time in four keeps about half of what it
# could give. Checking the inputs is worth more than adding
# dimensions to an uncertain profile.The gain falls faster than the error rate rises: at an error rate of 0.10 it keeps 0.7878, at 0.25 only 0.5350, at 0.50 only 0.2363. Checking the profile’s inputs is worth more than adding dimensions to an unreliable profile.
Hypothesis · not established
The author’s interpretation (not yet tested): in the author’s system the five-dimensional profile really does change the distribution of fitting advice (), and with inputs such as birth time checked, the profile’s error rate is below 0.10. What would show this wrong: if, in the blind comparison of Chapter 8, users showed no clear preference between answers generated with their own profile and with a stranger’s profile swapped in, would be close to zero in practice.
Summary
What a profile does for answers is measured by conditional mutual information, which is bounded by the uncertainty of the answer and by the answer’s dependence on the profile, not by the size of the file: in this book’s example a profile of 7.9248 bits brings a gain of 0.8519 bits, and 0 when answers ignore it. With a quarter of the profile wrong, only 0.5350 of the gain is left.
References
Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents: Multidimensional State Vectors, Personalized Prompt Conditioning and Cross-Corpus Synthesis [五维灵商量化架构、先验条件化生成与检索增强认知体] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).