Laplace’s demon and Lorenz’s computer

If the universe obeys physical law strictly, an intellect that knew the positions and velocities of all particles at this moment could compute the whole future: this is the deterministic picture Laplace set out in the early nineteenth century. Many people infer from it that, since everything is already fixed, there is no such thing as changing one’s fate.

In 1963 the meteorologist Edward Lorenz found something in a set of simplified convection equations with only three variables: the equations are entirely deterministic, with no random term, yet two solutions starting almost exactly together head off in completely different directions after a while (Lorenz 1963). This came to be called sensitive dependence on initial conditions, popularly the “butterfly effect”.

$$\dot{x} = \sigma(y-x),\qquad \dot{y} = x(\rho – z) – y,\qquad \dot{z} = xy – \beta z,$$

with $\sigma = 10$, $\rho = 28$, $\beta = 8/3$.

# The Lorenz (1963) system, sigma = 10, rho = 28, beta = 8/3, integrated with fourth-order Runge-Kutta.
# Two runs start 1e-9 apart. Their separation grows roughly as exp(lambda t); lambda (the largest
# Lyapunov exponent) is estimated by repeatedly renormalising a nearby trajectory (Benettin's method).
import math
S, R, B = 10.0, 28.0, 8.0 / 3.0

def deriv(p):
    x, y, z = p
    return (S * (y - x), x * (R - z) - y, x * y - B * z)

def rk4(p, h):
    k1 = deriv(p)
    k2 = deriv([p[i] + h / 2 * k1[i] for i in range(3)])
    k3 = deriv([p[i] + h / 2 * k2[i] for i in range(3)])
    k4 = deriv([p[i] + h * k3[i] for i in range(3)])
    return [p[i] + h / 6 * (k1[i] + 2 * k2[i] + 2 * k3[i] + k4[i]) for i in range(3)]

def dist(p, q):
    return math.sqrt(sum((a - b) ** 2 for a, b in zip(p, q)))

H = 0.01
a = [1.0, 1.0, 20.0]
for _ in range(2000):                      # settle onto the attractor
    a = rk4(a, H)
b = [a[0] + 1e-9, a[1], a[2]]
print("time | separation of the two runs")
for step in range(1, 4001):
    a, b = rk4(a, H), rk4(b, H)
    if step % 500 == 0:
        print(f"{step * H:4.0f} | {dist(a, b):.2e}")

p = a[:]; q = [p[0] + 1e-8, p[1], p[2]]; total = 0.0; T = 0
for step in range(1, 100001):
    p, q = rk4(p, H), rk4(q, H)
    if step % 10 == 0:
        d = dist(p, q); total += math.log(d / 1e-8); T += 10 * H
        q = [p[i] + (q[i] - p[i]) * 1e-8 / d for i in range(3)]
lam = total / T
print(f"largest Lyapunov exponent ~ {lam:.3f} per unit time")
for err in (1e-9, 1e-6, 1e-3):
    print(f"initial error {err:.0e}: grows to size 1 after about {math.log(1 / err) / lam:4.1f} time units")
# time | separation of the two runs
#    5 | 6.16e-09
#   10 | 2.70e-07
#   15 | 2.47e-05
#   20 | 1.44e-03
#   25 | 4.96e-02
#   30 | 1.02e+01
#   35 | 2.14e+01
#   40 | 8.30e+00
# largest Lyapunov exponent ~ 0.917 per unit time
# initial error 1e-09: grows to size 1 after about 22.6 time units
# initial error 1e-06: grows to size 1 after about 15.1 time units
# initial error 1e-03: grows to size 1 after about  7.5 time units

The separation of the two trajectories grows by a factor of about $e^{0.92} \approx 2.5$ per time unit. The program’s estimate of the largest Lyapunov exponent, about 0.92, is close to the commonly quoted value of about 0.906. This gives a predictability horizon: an initial error of $10^{-9}$ grows to the size of the attractor in about 23 time units, an error of $10^{-3}$ in only about 7.5. Each thousandfold improvement in initial measurement adds only about 7.5 time units of prediction: better precision buys only a linear extension of foresight.

What chaos shows

Chaotic dynamics gives this book a useful but limited analogy.

It shows that determinism is not predictability. Even when a system obeys fixed rules completely, knowledge of finite precision supports prediction only for a finite time. Over a long enough time scale, the course of a trajectory that looks “fated” depends on tiny present differences.

It also shows that sensitivity cuts both ways. Small differences can grow into large ones, but the direction of growth is not under control: in a chaotic system a tiny change can push the trajectory toward a better region or a worse one, and there is no way of knowing in advance. Reading the butterfly effect as “a small good intention will surely bring a large reward” is a misreading.

Compatibilism

On determinism and free will, philosophy has a long-standing position called compatibilism: freedom lies not in actions having no cause but in their being caused by the agent’s own reasons, desires and reflection. Frankfurt argued that a person’s freedom of the will lies in being able to form “second-order desires”, desires about which desires to have (Frankfurt 1971); Dennett argued that the kinds of freedom worth wanting are entirely compatible with determinism (Dennett 1984).

In this book’s framework, compatibilism can be understood like this. A person’s priors and patterns of behaviour are shaped by the past; that is the deterministic side. But the person’s generative model also contains a model of themselves, which can take their own patterns as objects of observation and change action accordingly. That change also has causes: awareness of themselves and new experience. The Chapter 2 agent that kept opening in the sandbox was also caused to do so, yet the change altered its whole subsequent trajectory.

Summary

Chaos does not give us freedom, but it removes a common fatalist argument: “the world is deterministic, so the future is fixed, so my choices do not matter.” In a world sensitive to its present state, the present state includes ourselves and what we understand about ourselves.