Chapter 2 Two Routes: Change the Mind, or Change the World

Variational free energy

Exact Bayesian inference is often impossible to compute: the posterior needs the probability of the observation itself, $p(o)$ , which means summing over every possible hidden cause. The variational approach does not compute the posterior directly. It looks instead for a tractable approximate distribution $q(s)$ that comes as close to the posterior as possible. The measure of closeness is the variational free energy:

$F[q] = \underbrace{D_{\mathrm{KL}}\big[q(s)\,\|\,p(s)\big]}_{\text{complexity}} – \underbrace{\mathbb{E}_{q}\big[\ln p(o\mid s)\big]}_{\text{accuracy}} \;\ge\; -\ln p(o).$

The term $-\ln p(o)$ on the right is the surprise: the more unexpected an observation is under the model, the larger it is. Free energy is never smaller than surprise and equals it when $q$ is exactly the true posterior. Minimising free energy therefore does two things at once: it brings the approximate belief close to the posterior, and it gives a computable upper bound on surprise (Friston 2010; Parr et al. 2022).

Calculation

A two-state world: the other person is “safe” or “hostile”, prior 3:7; one friendly reply is observed.

# Variational free energy for a two-state world: F(q) = KL[q(s) || p(s)] - E_q[ln p(o|s)].
# F is never below the surprise -ln p(o); it equals it when q is the exact posterior.
import math
prior = {"safe": 0.3, "hostile": 0.7}          # p(s): the agent's prior
lik = {"safe": 0.8, "hostile": 0.2}            # p(o = 'friendly reply' | s)

p_o = sum(prior[s] * lik[s] for s in prior)
post_safe = prior["safe"] * lik["safe"] / p_o
print(f"surprise -ln p(o) = {-math.log(p_o):.4f} nats; exact posterior q(safe) = {post_safe:.3f}")

def F(q_safe):
    q = {"safe": q_safe, "hostile": 1 - q_safe}
    kl = sum(q[s] * math.log(q[s] / prior[s]) for s in q if q[s] > 0)
    acc = sum(q[s] * math.log(lik[s]) for s in q)
    return kl - acc, kl, acc

for qs in (0.05, 0.30, post_safe, 0.80, 0.95):
    f, kl, acc = F(qs)
    print(f"q(safe) = {qs:.3f}: complexity {kl:.4f}, accuracy {acc:+.4f}, F = complexity - accuracy = {f:.4f}")
best = min((F(i / 1000)[0], i / 1000) for i in range(1, 1000))
print(f"grid minimum of F at q(safe) = {best[1]:.3f}, F = {best[0]:.4f}")
# surprise -ln p(o) = 0.9676 nats; exact posterior q(safe) = 0.632
# q(safe) = 0.050: complexity 0.2005, accuracy -1.5401, F = complexity - accuracy = 1.7406
# q(safe) = 0.300: complexity 0.0000, accuracy -1.1935, F = complexity - accuracy = 1.1935
# q(safe) = 0.632: complexity 0.2337, accuracy -0.7339, F = complexity - accuracy = 0.9676
# q(safe) = 0.800: complexity 0.5341, accuracy -0.5004, F = complexity - accuracy = 1.0345
# q(safe) = 0.950: complexity 0.9631, accuracy -0.2925, F = complexity - accuracy = 1.2556
# grid minimum of F at q(safe) = 0.632, F = 0.9676

The complexity term measures how far a belief departs from the prior; the accuracy term measures how well the belief explains the observation. Leaving the belief at the prior ( $q = 0.30$ ) costs no complexity but explains the observation poorly; pushing it too far ( $q = 0.95$ ) explains it well at a high cost in complexity. The minimum lies at the exact posterior, 0.632, where free energy equals the surprise of 0.9676 nats.

The second route: action

So far there is only one way to reduce free energy: change the belief. Active inference adds a second: action changes observations. If an outcome is already “expected” by a person, they can act so that the outcome occurs, and the prediction error shrinks (Friston 2010; Da Costa et al. 2020).

In discrete active-inference models the agent computes the expected free energy $G$ of each available action (or sequence of actions, a “policy”) and chooses with $P(\pi) \propto \exp(-\gamma\,G(\pi))$ . Expected free energy is often split into two terms (Friston et al. 2015; Da Costa et al. 2020):

$G(\pi) = \underbrace{D_{\mathrm{KL}}\big[q(o\mid\pi)\,\|\,p_C(o)\big]}_{\text{risk}} + \underbrace{\mathbb{E}_{q(s\mid\pi)}\big[H[p(o\mid s)]\big]}_{\text{ambiguity}}.$

The key is $p_C(o)$ . In active inference the agent’s “preferences” are not separately specified rewards but prior beliefs about outcomes: what it expects to encounter. The risk term measures how far the outcomes an action is expected to bring lie from this prior; the ambiguity term measures how hard the outcomes are to predict.

Note

Writing preferences as priors is a formal feature of active inference. It is mathematically exact, but equating it with a person’s expectations, premonitions or self-concept is an interpretation. The model that follows rests on that interpretation and is therefore a simplified model, not a direct measurement of human psychology.

An agent caught in a trap

The program below builds a minimal example. In each round the agent meets a stranger who is warm or cool, half and half. It can “open” (reach out) or “guard” (be pre-emptively cold). When it opens, a warm person accepts it nine times in ten and a cool person six times in ten; when it guards, the chances of acceptance are only 20 % and 5 %. Objectively, opening is clearly better: always opening is accepted 75 % of the time, always guarding 12.5 %.

Calculation

Expected free energy, its two terms, and the probability of opening under three outcome priors; then the outcome prior is learned from experience (counts with forgetting) over 200 simulated lives.

# A minimal active-inference agent caught in a self-confirming loop (a simplified model).
# Two actions: 'open' (reach out) and 'guard' (pre-emptive coldness). Two outcomes: accepted, rejected.
# Hidden state: the other person is warm or cool (prior 50:50). In active inference the agent's
# preferences ARE prior beliefs about outcomes, C = p(o). Expected free energy of an action:
#   G = risk + ambiguity = KL[q(o|a) || C] + E_q(s)[ H[p(o|s,a)] ]
import math, random
A = {"open":  {"warm": 0.90, "cool": 0.60},      # p(accepted | state, action)
     "guard": {"warm": 0.20, "cool": 0.05}}
Q_S = {"warm": 0.5, "cool": 0.5}

def H(p):
    return -sum(x * math.log(x) for x in (p, 1 - p) if x > 0)

def G(action, c_acc):
    qa = sum(Q_S[s] * A[action][s] for s in Q_S)               # predicted p(accepted)
    risk = qa * math.log(qa / c_acc) + (1 - qa) * math.log((1 - qa) / (1 - c_acc))
    ambiguity = sum(Q_S[s] * H(A[action][s]) for s in Q_S)
    return risk, ambiguity

def p_open(c_acc, gamma=4.0):
    g = {a: sum(G(a, c_acc)) for a in A}
    z = {a: math.exp(-gamma * g[a]) for a in A}
    return z["open"] / (z["open"] + z["guard"])

for c in (0.2, 0.5, 0.8):
    ro, ao = G("open", c); rg, ag = G("guard", c)
    print(f"C(accepted) = {c}: G(open) = {ro:.3f}+{ao:.3f} = {ro+ao:.3f}, "
          f"G(guard) = {rg:.3f}+{ag:.3f} = {rg+ag:.3f}, P(open) = {p_open(c):.2f}")

def life(c0, trials=400, forget=0.99, n0=10.0, forced_open=0, seed=1):
    """C is learned: it tracks the outcomes the agent has experienced (Dirichlet counts with forgetting).
    forced_open = number of early low-stakes trials in which the agent tries 'open' regardless."""
    rng = random.Random(seed)
    acc, rej = c0 * n0, (1 - c0) * n0
    opens = []
    for t in range(trials):
        c = min(max(acc / (acc + rej), 0.01), 0.99)
        a = "open" if t < forced_open or rng.random() < p_open(c) else "guard"
        s = "warm" if rng.random() < 0.5 else "cool"
        accepted = rng.random() < A[a][s]
        acc = forget * acc + accepted
        rej = forget * rej + (not accepted)
        opens.append(a == "open")
    return sum(opens[-100:]) / 100, acc / (acc + rej)

print("the same world: always 'open' is accepted", sum(Q_S[s] * A["open"][s] for s in Q_S),
      "of the time, always 'guard'", sum(Q_S[s] * A["guard"][s] for s in Q_S))
print("learned preferences, last 100 of 400 trials (200 simulated lives each):")
for label, c0, f in (("hopeful start C=0.8", 0.8, 0), ("bleak start C=0.2", 0.2, 0),
                     ("bleak start + 10 sandbox trials of 'open'", 0.2, 10),
                     ("bleak start + 30 sandbox trials of 'open'", 0.2, 30)):
    res = [life(c0, forced_open=f, seed=k) for k in range(200)]
    po = sum(r[0] for r in res) / len(res)
    esc = sum(r[0] > 0.5 for r in res) / len(res)
    print(f"  {label:42s} P(open) {po:.2f}, share of lives mostly open {esc:.0%}")
# C(accepted) = 0.2: G(open) = 0.701+0.499 = 1.200, G(guard) = 0.020+0.349 = 0.369, P(open) = 0.03
# C(accepted) = 0.5: G(open) = 0.131+0.499 = 0.630, G(guard) = 0.316+0.349 = 0.666, P(open) = 0.54
# C(accepted) = 0.8: G(open) = 0.007+0.499 = 0.506, G(guard) = 1.059+0.349 = 1.409, P(open) = 0.97
# the same world: always 'open' is accepted 0.75 of the time, always 'guard' 0.125
# learned preferences, last 100 of 400 trials (200 simulated lives each):
#   hopeful start C=0.8                        P(open) 0.75, share of lives mostly open 85%
#   bleak start C=0.2                          P(open) 0.01, share of lives mostly open 0%
#   bleak start + 10 sandbox trials of 'open'  P(open) 0.16, share of lives mostly open 16%
#   bleak start + 30 sandbox trials of 'open'  P(open) 0.62, share of lives mostly open 70%

Take the first part first. When the agent’s outcome prior is “rejection” (acceptance only 0.2), the risk of opening is large (0.701), because opening would bring too much acceptance that it “does not expect”; the risk of guarding is almost zero, because guarding brings exactly the rejection it expects. Add the fact that guarding’s outcome is more predictable (lower ambiguity), and it opens only 3 % of the time. In the same world, an agent whose outcome prior is 0.8 opens 97 % of the time.

The second part lets the outcome prior learn from experience: the agent expects what it has recently experienced. Two separately stable states appear. Agents that start hopeful still mostly open after 400 rounds in 85 % of lives; agents that start bleak are stuck guarding in every life: they guard, so they are rejected, so they expect rejection more, so they guard more. This is the computational version of the “fate” in this book’s subtitle: not an external destiny, but a self-sustaining loop.

The probability of choosing “open” over trials (each curve is the mean of 200 simulations). Hopeful and bleak starts go to two different stable states in the same world; after a bleak start, opening regardless for the first 30 trials under low stakes frees most simulations from the trap.

The last two lines answer what can break the loop. If in the first rounds the agent tries opening under low stakes regardless of its expectation (Chapter 5 discusses such “sandboxes”), it accumulates experiences of acceptance. Ten rounds are not enough: only 16 % of simulations escape. After 30 rounds, 70 % move to a state of mostly opening. There is a threshold: experience must push the outcome prior past an unstable dividing point, or the loop pulls it back.

The model also reveals a less obvious point: the ambiguity term on its own favours guarding, because guarding’s outcome is easier to predict. Even for hopeful agents the curve slowly declines: an occasional run of rejections pushes them toward the trap, and once in, it is hard to leave. In this model, part of the price of openness is tolerating more uncertainty.

When the two routes part

In the model above the choice of action is discrete. The program below uses a continuous simplified model that can be solved exactly, and asks a more general question: faced with the same prediction error, what share does a system remove by changing its belief, and what share by changing the world?

Calculation

Minimise $F = \tfrac{\pi_o}{2}(o-\mu)^2 + \tfrac{\pi_p}{2}(\mu-m)^2 + \tfrac{k}{2}(o-o_0)^2$ , where $\mu$ is the belief, $m$ the prior, $o$ the observation, $o_0$ the outcome the world would deliver if left alone, and $k$ the cost of changing the world by action.

# Two ways to cancel a prediction error (a linear-quadratic simplified model).
# Prior belief m, current estimate mu, observed outcome o. The world would deliver o0 if left alone;
# action can move o away from o0 at a cost k. Minimise
#   F = pi_o/2 (o - mu)^2 + pi_p/2 (mu - m)^2 + k/2 (o - o0)^2   over mu (perception) and o (action).
# Setting both derivatives to zero gives a 2x2 linear system, solved exactly below.
def solve(pi_o, pi_p, k, m=-2.0, o0=1.0):
    # dF/dmu: -pi_o (o - mu) + pi_p (mu - m) = 0 ; dF/do: pi_o (o - mu) + k (o - o0) = 0
    a11, a12, b1 = pi_o + pi_p, -pi_o, pi_p * m
    a21, a22, b2 = -pi_o, pi_o + k, k * o0
    det = a11 * a22 - a12 * a21
    mu = (b1 * a22 - a12 * b2) / det
    o = (a11 * b2 - a21 * b1) / det
    return mu, o

M, O0 = -2.0, 1.0
print("initial error between prior and world:", O0 - M)
print("prior precision  action cost | belief moved  world moved  residual | share removed by action")
for pi_p, k in ((0.5, 4.0), (2.0, 4.0), (8.0, 4.0), (8.0, 1.0), (32.0, 1.0)):
    mu, o = solve(1.0, pi_p, k)
    belief_shift, world_shift = mu - M, O0 - o
    share = world_shift / (belief_shift + world_shift)
    print(f"{pi_p:15.1f} {k:12.1f} | {belief_shift:+12.2f} {-world_shift:+12.2f} {o - mu:+9.2f} | {share:6.0%}")
# initial error between prior and world: 3.0
# prior precision  action cost | belief moved  world moved  residual | share removed by action
#             0.5          4.0 |        +1.85        -0.23     +0.92 |    11%
#             2.0          4.0 |        +0.86        -0.43     +1.71 |    33%
#             8.0          4.0 |        +0.27        -0.55     +2.18 |    67%
#             8.0          1.0 |        +0.18        -1.41     +1.41 |    89%
#            32.0          1.0 |        +0.05        -1.48     +1.48 |    97%

With a low-precision prior, the error is removed mainly by changing the belief (action carries only 11 %). With a high-precision prior and a low cost of acting, action removes 97 % of the error: the belief barely moves and the world is pushed toward the prediction. This is the mathematical form of the “two routes”. It also explains why argument so often fails: for a system with a very precise prior, the cheapest move is not to change its view but to change its circumstances until they fit the view again.

Repetition compulsion: an analogy to handle with care

In 1920 Freud described what he called the “compulsion to repeat”: people repeatedly put themselves into painful situations, as if driven by some force (Freud 1920). In recent years computational psychiatry has tried to restate a range of psychopathology in the language of predictive processing and active inference, for example describing some symptoms as abnormal prior precision or abnormal preference structure (Smith et al. 2021; Huys et al. 2016).

Hypothesis · not established

Not established. Some recurring patterns of interpersonal setback can be partly described as a precise, pessimistic outcome prior that, through active inference, drives the actions that confirm it. The hypothesis predicts that (1) among people who repeatedly experience the same kind of setback, outcome priors fitted to behavioural data by a computational model are more pessimistic and more precise than in a comparison group, and (2) contrary experience accumulated under low stakes shifts behavioural choice near the threshold the model predicts. If the fitted priors are unrelated to the setback pattern, or behaviour does not change once experience has accumulated past the model’s threshold, the hypothesis is refuted.

Note

The limits of this analogy. First, it is a hypothesis, not a diagnosis, and not an explanation of any particular person. Second, many harms — abuse, violence, discrimination, accident, illness — are simply not caused by the people who suffer them; no theory may be used to suggest that victims “brought on” what happened to them. The model concerns one specific situation: the same person making the same choice again and again in situations that could have turned out differently. Third, the effects of childhood trauma are complex and far-reaching and need qualified professional help; this book cannot and does not try to replace psychotherapy. Fourth, “fate” here is only a folk word: the model explains recurring patterns, not destiny in the literal sense.

References

Da Costa, Lancelot, Thomas Parr, Noor Sajid, Sebastijan Veselic, Victorita Neacsu, and Karl Friston. 2020. “Active Inference on Discrete State-Spaces: A Synthesis.” Journal of Mathematical Psychology 99: 102447.

Freud, Sigmund. 1920. Jenseits Des Lustprinzips. Internationaler Psychoanalytischer Verlag.

Friston, Karl. 2010. “The Free-Energy Principle: A Unified Brain Theory?” Nature Reviews Neuroscience 11 (2): 127–38.

Friston, Karl, Francesco Rigoli, Dimitri Ognibene, Christoph Mathys, Thomas Fitzgerald, and Giovanni Pezzulo. 2015. “Active Inference and Epistemic Value.” Cognitive Neuroscience 6 (4): 187–214.

Huys, Quentin J. M., Tiago V. Maia, and Michael J. Frank. 2016. “Computational Psychiatry as a Bridge from Neuroscience to Clinical Applications.” Nature Neuroscience 19 (3): 404–13.

Parr, Thomas, Giovanni Pezzulo, and Karl J. Friston. 2022. Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. MIT Press.

Smith, Ryan, Paul Badcock, and Karl J. Friston. 2021. “Recent Advances in the Application of Predictive Coding and Active Inference Models Within Clinical Neuroscience.” Psychiatry and Clinical Neurosciences 75 (1): 3–13.

Fan, Y. (2026). The Sandbox, Determinism and Active Inference: Markov Blankets, Variational Free-Energy Minimisation, Exploration-Exploitation Trade-offs and East Asian Causality in Computational Neuroscience [沙盘、宿命与主动推理] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23738794 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).