5D Library

第二章 两条路径：改变认知，还是改变世界

5D Library › 沙盘、宿命与主动推理 · English

第二章 两条路径：改变认知，还是改变世界

变分自由能

精确的贝叶斯推断往往算不出来：要算后验，需要知道观察本身的概率 p(o)p(o)，而这需要对所有可能的隐藏原因求和。变分方法的思路是：不直接求后验，而是找一个容易处理的近似分布 q(s)q(s)，让它尽量接近后验。衡量「接近」的量就是变分自由能：

F[q]=DKL[q(s)∥p(s)]⏟复杂度−𝔼q[ln⁡p(o∣s)]⏟准确度≥−ln⁡p(o).F[q] = \underbrace{D_{\mathrm{KL}}\big[q(s)\,\|\,p(s)\big]}_{\text{复杂度}} – \underbrace{\mathbb{E}_{q}\big[\ln p(o\mid s)\big]}_{\text{准确度}} \;\ge\; -\ln p(o).

右边的 −ln⁡p(o)-\ln p(o) 叫作惊奇：观察越出乎模型的意料，它越大。自由能永远不小于惊奇，当 qq 恰好等于真实后验时两者相等。所以最小化自由能同时做了两件事：使近似信念接近后验，并给惊奇提供一个可计算的上界 (Friston 2010; Parr et al. 2022)。

推导

一个两状态的世界：对方是「安全」还是「敌意」，先验是 3:7；观察到一次友好的回复。

# Variational free energy for a two-state world: F(q) = KL[q(s) || p(s)] - E_q[ln p(o|s)].
# F is never below the surprise -ln p(o); it equals it when q is the exact posterior.
import math
prior = {"safe": 0.3, "hostile": 0.7}          # p(s): the agent's prior
lik = {"safe": 0.8, "hostile": 0.2}            # p(o = 'friendly reply' | s)

p_o = sum(prior[s] * lik[s] for s in prior)
post_safe = prior["safe"] * lik["safe"] / p_o
print(f"surprise -ln p(o) = {-math.log(p_o):.4f} nats; exact posterior q(safe) = {post_safe:.3f}")

def F(q_safe):
    q = {"safe": q_safe, "hostile": 1 - q_safe}
    kl = sum(q[s] * math.log(q[s] / prior[s]) for s in q if q[s] > 0)
    acc = sum(q[s] * math.log(lik[s]) for s in q)
    return kl - acc, kl, acc

for qs in (0.05, 0.30, post_safe, 0.80, 0.95):
    f, kl, acc = F(qs)
    print(f"q(safe) = {qs:.3f}: complexity {kl:.4f}, accuracy {acc:+.4f}, F = complexity - accuracy = {f:.4f}")
best = min((F(i / 1000)[0], i / 1000) for i in range(1, 1000))
print(f"grid minimum of F at q(safe) = {best[1]:.3f}, F = {best[0]:.4f}")
# surprise -ln p(o) = 0.9676 nats; exact posterior q(safe) = 0.632
# q(safe) = 0.050: complexity 0.2005, accuracy -1.5401, F = complexity - accuracy = 1.7406
# q(safe) = 0.300: complexity 0.0000, accuracy -1.1935, F = complexity - accuracy = 1.1935
# q(safe) = 0.632: complexity 0.2337, accuracy -0.7339, F = complexity - accuracy = 0.9676
# q(safe) = 0.800: complexity 0.5341, accuracy -0.5004, F = complexity - accuracy = 1.0345
# q(safe) = 0.950: complexity 0.9631, accuracy -0.2925, F = complexity - accuracy = 1.2556
# grid minimum of F at q(safe) = 0.632, F = 0.9676

复杂度项衡量信念偏离先验多远，准确度项衡量信念对观察解释得多好。把信念留在先验上（q=0.30q = 0.30）复杂度为零，但解释观察很差；把信念推得过头（q=0.95q = 0.95）解释得好，代价是复杂度太高。最小值落在精确后验 0.632 处，这时自由能等于惊奇 0.9676 奈特。

第二条路：行动

到这里为止，减小自由能只有一条路：修改信念。主动推理补上了第二条路：行动改变观察。如果对一个人来说，某种结果已经被「预期」了，那么他可以通过行动让这种结果出现，从而使预测误差变小 (Friston 2010; Da Costa et al. 2020)。

在离散的主动推理模型里，智能体对每一种可选行动（或行动序列，即「策略」）计算期望自由能 GG，并按 P(π)∝exp⁡(−γG(π))P(\pi) \propto \exp(-\gamma\,G(\pi)) 选择。期望自由能常被分解为两项 (Friston et al. 2015; Da Costa et al. 2020)：

G(π)=DKL[q(o∣π)∥pC(o)]⏟风险+𝔼q(s∣π)[H[p(o∣s)]]⏟模糊性.G(\pi) = \underbrace{D_{\mathrm{KL}}\big[q(o\mid\pi)\,\|\,p_C(o)\big]}_{\text{风险}} + \underbrace{\mathbb{E}_{q(s\mid\pi)}\big[H[p(o\mid s)]\big]}_{\text{模糊性}}.

这里的关键是 pC(o)p_C(o)。在主动推理中，智能体的「偏好」不是另外设定的奖励，而是关于结果的先验信念：它预期自己会遇到什么。风险项衡量一种行动预计带来的结果与这一先验的差距；模糊性项衡量结果有多难预测。

注

这种「偏好即先验」的写法是主动推理的形式特征。它在数学上是严格的，但把它等同于人的「期望」「预感」或「自我概念」，是一种诠释。本章接下来的模型就建立在这种诠释上，因此是简化模型，而不是对人类心理的直接测量。

一个困在陷阱里的智能体

下面的程序构造一个最小的例子。智能体每一轮遇到一个陌生人，对方的性情温暖或冷淡，各占一半。它可以选择「敞开」（主动接触）或「设防」（先发制人地冷淡）。敞开时，温暖的人九成会接纳它，冷淡的人六成会接纳；设防时，接纳的机会只有 20% 和 5%。客观上，敞开明显更好：一直敞开会有 75% 的时间被接纳，一直设防只有 12.5%。

推导

期望自由能、它的两项，以及在三种结果先验下选择「敞开」的概率；然后让结果先验随经验学习（带遗忘的计数），模拟 200 次「人生」。

# A minimal active-inference agent caught in a self-confirming loop (a simplified model).
# Two actions: 'open' (reach out) and 'guard' (pre-emptive coldness). Two outcomes: accepted, rejected.
# Hidden state: the other person is warm or cool (prior 50:50). In active inference the agent's
# preferences ARE prior beliefs about outcomes, C = p(o). Expected free energy of an action:
#   G = risk + ambiguity = KL[q(o|a) || C] + E_q(s)[ H[p(o|s,a)] ]
import math, random
A = {"open":  {"warm": 0.90, "cool": 0.60},      # p(accepted | state, action)
     "guard": {"warm": 0.20, "cool": 0.05}}
Q_S = {"warm": 0.5, "cool": 0.5}

def H(p):
    return -sum(x * math.log(x) for x in (p, 1 - p) if x > 0)

def G(action, c_acc):
    qa = sum(Q_S[s] * A[action][s] for s in Q_S)               # predicted p(accepted)
    risk = qa * math.log(qa / c_acc) + (1 - qa) * math.log((1 - qa) / (1 - c_acc))
    ambiguity = sum(Q_S[s] * H(A[action][s]) for s in Q_S)
    return risk, ambiguity

def p_open(c_acc, gamma=4.0):
    g = {a: sum(G(a, c_acc)) for a in A}
    z = {a: math.exp(-gamma * g[a]) for a in A}
    return z["open"] / (z["open"] + z["guard"])

for c in (0.2, 0.5, 0.8):
    ro, ao = G("open", c); rg, ag = G("guard", c)
    print(f"C(accepted) = {c}: G(open) = {ro:.3f}+{ao:.3f} = {ro+ao:.3f}, "
          f"G(guard) = {rg:.3f}+{ag:.3f} = {rg+ag:.3f}, P(open) = {p_open(c):.2f}")

def life(c0, trials=400, forget=0.99, n0=10.0, forced_open=0, seed=1):
    """C is learned: it tracks the outcomes the agent has experienced (Dirichlet counts with forgetting).
    forced_open = number of early low-stakes trials in which the agent tries 'open' regardless."""
    rng = random.Random(seed)
    acc, rej = c0 * n0, (1 - c0) * n0
    opens = []
    for t in range(trials):
        c = min(max(acc / (acc + rej), 0.01), 0.99)
        a = "open" if t < forced_open or rng.random() < p_open(c) else "guard"
        s = "warm" if rng.random() < 0.5 else "cool"
        accepted = rng.random() < A[a][s]
        acc = forget * acc + accepted
        rej = forget * rej + (not accepted)
        opens.append(a == "open")
    return sum(opens[-100:]) / 100, acc / (acc + rej)

print("the same world: always 'open' is accepted", sum(Q_S[s] * A["open"][s] for s in Q_S),
      "of the time, always 'guard'", sum(Q_S[s] * A["guard"][s] for s in Q_S))
print("learned preferences, last 100 of 400 trials (200 simulated lives each):")
for label, c0, f in (("hopeful start C=0.8", 0.8, 0), ("bleak start C=0.2", 0.2, 0),
                     ("bleak start + 10 sandbox trials of 'open'", 0.2, 10),
                     ("bleak start + 30 sandbox trials of 'open'", 0.2, 30)):
    res = [life(c0, forced_open=f, seed=k) for k in range(200)]
    po = sum(r[0] for r in res) / len(res)
    esc = sum(r[0] > 0.5 for r in res) / len(res)
    print(f"  {label:42s} P(open) {po:.2f}, share of lives mostly open {esc:.0%}")
# C(accepted) = 0.2: G(open) = 0.701+0.499 = 1.200, G(guard) = 0.020+0.349 = 0.369, P(open) = 0.03
# C(accepted) = 0.5: G(open) = 0.131+0.499 = 0.630, G(guard) = 0.316+0.349 = 0.666, P(open) = 0.54
# C(accepted) = 0.8: G(open) = 0.007+0.499 = 0.506, G(guard) = 1.059+0.349 = 1.409, P(open) = 0.97
# the same world: always 'open' is accepted 0.75 of the time, always 'guard' 0.125
# learned preferences, last 100 of 400 trials (200 simulated lives each):
#   hopeful start C=0.8                        P(open) 0.75, share of lives mostly open 85%
#   bleak start C=0.2                          P(open) 0.01, share of lives mostly open 0%
#   bleak start + 10 sandbox trials of 'open'  P(open) 0.16, share of lives mostly open 16%
#   bleak start + 30 sandbox trials of 'open'  P(open) 0.62, share of lives mostly open 70%

先看第一部分。当智能体的结果先验是「被拒绝」（接纳只占 0.2）时，敞开的风险项很大（0.701），因为敞开会带来太多它「没有预期到」的接纳；设防的风险几乎为零，因为设防恰好带来它预期的拒绝。再加上设防的结果更可预测（模糊性更低），它选择敞开的概率只有 3%。同一个世界里，结果先验是 0.8 的智能体则 97% 选择敞开。

第二部分让结果先验从经验中学习：智能体预期的，就是它最近经历的。于是出现两个各自稳定的状态。乐观起步的智能体在 400 轮后仍有 85% 的「人生」以敞开为主；悲观起步的则 100% 陷在设防里：它设防，于是被拒绝，于是更预期被拒绝，于是更设防。这正是本书副标题里「宿命」的计算版本：不是外在的命运，而是一个自我维持的循环。

选择「敞开」的概率随轮次的变化（每条曲线为 200 次模拟的平均）。乐观起步与悲观起步在同一个世界里走向两个不同的稳定状态；悲观起步后，前 30 轮在低风险条件下坚持敞开，可使大多数模拟脱离陷阱。

最后两行回答「什么能打破循环」。如果在最初若干轮里，智能体在低风险的条件下不管预期如何都尝试敞开（第五章将讨论这种「沙盘」），它就会积累被接纳的经验。10 轮还不够，只有 16% 的模拟脱离陷阱；30 轮后，70% 的模拟转入以敞开为主的状态。这里有一个阈值：经验必须足够把结果先验推过不稳定的分界点，否则循环会把它拉回去。

模型还揭示了一个不那么显眼的结论：模糊性项本身就偏向设防，因为设防的结果更好预测。即使是乐观起步的智能体，曲线也在缓慢下降：偶然连续的几次拒绝会把它推向陷阱，而陷阱一旦进入就很难离开。在这个模型里，敞开的代价之一，就是要容忍更大的不确定性。

两条路径何时分道

上面的模型里，行动的选择是离散的。下面的程序换一个连续的、可以精确求解的简化模型，问一个更一般的问题：面对同样大小的预测误差，系统会用多大比例去修改信念，用多大比例去改变世界？

推导

最小化 F=πo2(o−μ)2+πp2(μ−m)2+k2(o−o0)2F = \tfrac{\pi_o}{2}(o-\mu)^2 + \tfrac{\pi_p}{2}(\mu-m)^2 + \tfrac{k}{2}(o-o_0)^2：μ\mu 是信念，mm 是先验，oo 是观察，o0o_0 是世界本来会给出的结果，kk 是用行动改变世界的成本。

# Two ways to cancel a prediction error (a linear-quadratic simplified model).
# Prior belief m, current estimate mu, observed outcome o. The world would deliver o0 if left alone;
# action can move o away from o0 at a cost k. Minimise
#   F = pi_o/2 (o - mu)^2 + pi_p/2 (mu - m)^2 + k/2 (o - o0)^2   over mu (perception) and o (action).
# Setting both derivatives to zero gives a 2x2 linear system, solved exactly below.
def solve(pi_o, pi_p, k, m=-2.0, o0=1.0):
    # dF/dmu: -pi_o (o - mu) + pi_p (mu - m) = 0 ; dF/do: pi_o (o - mu) + k (o - o0) = 0
    a11, a12, b1 = pi_o + pi_p, -pi_o, pi_p * m
    a21, a22, b2 = -pi_o, pi_o + k, k * o0
    det = a11 * a22 - a12 * a21
    mu = (b1 * a22 - a12 * b2) / det
    o = (a11 * b2 - a21 * b1) / det
    return mu, o

M, O0 = -2.0, 1.0
print("initial error between prior and world:", O0 - M)
print("prior precision  action cost | belief moved  world moved  residual | share removed by action")
for pi_p, k in ((0.5, 4.0), (2.0, 4.0), (8.0, 4.0), (8.0, 1.0), (32.0, 1.0)):
    mu, o = solve(1.0, pi_p, k)
    belief_shift, world_shift = mu - M, O0 - o
    share = world_shift / (belief_shift + world_shift)
    print(f"{pi_p:15.1f} {k:12.1f} | {belief_shift:+12.2f} {-world_shift:+12.2f} {o - mu:+9.2f} | {share:6.0%}")
# initial error between prior and world: 3.0
# prior precision  action cost | belief moved  world moved  residual | share removed by action
#             0.5          4.0 |        +1.85        -0.23     +0.92 |    11%
#             2.0          4.0 |        +0.86        -0.43     +1.71 |    33%
#             8.0          4.0 |        +0.27        -0.55     +2.18 |    67%
#             8.0          1.0 |        +0.18        -1.41     +1.41 |    89%
#            32.0          1.0 |        +0.05        -1.48     +1.48 |    97%

先验精度低时，误差主要由信念的修改来消除（行动只承担 11%）；先验精度高、改变世界的成本又低时，97% 的误差由行动来消除：信念几乎不动，世界被推向预测。这就是「两条路径」的数学形式。它也解释了为什么讲道理往往无效：对一个先验精度很高的系统，最省力的不是改变看法，而是改变处境，让处境重新符合看法。

强迫性重复：一个需要谨慎对待的类比

弗洛伊德在 1920 年描述过一种他称为「强迫性重复」的现象：人反复把自己置于痛苦的处境中，仿佛受某种力量驱使 (Freud 1920)。近年来，计算精神病学尝试用预测加工和主动推理的语言重新表述各种精神病理现象，例如把某些症状描述为先验精度异常或偏好结构异常 (Smith et al. 2021; Huys et al. 2016)。

假说 · 尚未验证

尚未确立。 部分反复出现的人际挫折模式，可以部分地描述为：一个精度很高、内容悲观的结果先验，通过主动推理驱动了使它自我证实的行动。该假说预测：(1) 在反复经历同类挫折的人中，用计算模型从行为数据拟合出的结果先验比对照组更悲观、精度更高；(2) 在低风险条件下累积的相反经验，能使行为选择在模型预测的阈值附近发生转变。如果拟合出的先验与挫折模式无关，或者在经验累积到模型所说的阈值以上之后行为并无变化，假说就被推翻。

注

这个类比的边界。 第一，它是假说，不是诊断，也不是对任何具体的人的解释。第二，许多伤害——虐待、暴力、歧视、意外、疾病——根本不是受害者引起的；任何理论都不能被用来暗示受害者「招来」了自己的遭遇。上面的模型只涉及一种特定的情形：同一个人在本可以有不同结果的情境中反复做出相同的选择。第三，童年创伤的影响复杂而深远，需要合格的专业人员帮助，本书不能也不打算替代心理治疗。第四，「宿命」在这里只是一个通俗说法：模型解释的是重复出现的模式，而不是字面意义上的命运。

参考文献

Da Costa, Lancelot, Thomas Parr, Noor Sajid, Sebastijan Veselic, Victorita Neacsu, and Karl Friston. 2020. “Active Inference on Discrete State-Spaces: A Synthesis.” Journal of Mathematical Psychology 99: 102447.
Freud, Sigmund. 1920. Jenseits Des Lustprinzips. Internationaler Psychoanalytischer Verlag.
Friston, Karl. 2010. “The Free-Energy Principle: A Unified Brain Theory?” Nature Reviews Neuroscience 11 (2): 127–38.
Friston, Karl, Francesco Rigoli, Dimitri Ognibene, Christoph Mathys, Thomas Fitzgerald, and Giovanni Pezzulo. 2015. “Active Inference and Epistemic Value.” Cognitive Neuroscience 6 (4): 187–214.
Huys, Quentin J. M., Tiago V. Maia, and Michael J. Frank. 2016. “Computational Psychiatry as a Bridge from Neuroscience to Clinical Applications.” Nature Neuroscience 19 (3): 404–13.
Parr, Thomas, Giovanni Pezzulo, and Karl J. Friston. 2022. Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. MIT Press.
Smith, Ryan, Paul Badcock, and Karl J. Friston. 2021. “Recent Advances in the Application of Predictive Coding and Active Inference Models Within Clinical Neuroscience.” Psychiatry and Clinical Neurosciences 75 (1): 3–13.

Fan, Y. (2026). The Sandbox, Determinism and Active Inference: Markov Blankets, Variational Free-Energy Minimisation, Exploration-Exploitation Trade-offs and East Asian Causality in Computational Neuroscience [沙盘、宿命与主动推理] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23738794 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).