What does The Limits of Guardrail Compression conclude, in numbers, about “Phase Transitions: Maximal Correlation, the Critical Exchange Rate and Deterministic Annealing”?

Answer: Chapter 4 shows that the critical exchange rate is the reciprocal of the squared maximal correlation, 0.6844 and 1.4610 on the illustrative source while information rises continuously as rules split.

Scope of the numbers: Chapter 1 builds an illustrative source of eight kinds of draft and three verdicts and the counting bound: three rules keep 0.960 of the verdict information, two only 0.629. Chapter 4 shows that the critical exchange rate is the reciprocal of the squared maximal correlation, 0.6844 and 1.4610 on the illustrative source while information rises continuously as rules split.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.6844, 1.4610

maximal correlation squared rho^2 = 0.6844
predicted critical beta 1/rho^2= 1.4610
beta/beta_cbetaI(X;T)I(T;Y)I(T;Y)/I(X;T)
0.901.31490.0000000.000000-
0.971.41720.0000000.000000-
1.031.50490.1217450.0824090.6769
1.101.60720.2820610.1857520.6586
1.251.82630.4677500.2949150.6305
1.502.19160.6101820.3669000.6013
Show the full program
# Chapter 4. p41: the first phase transition. Below a critical beta the
# best bottleneck says nothing (one rule for everything); above it a
# second rule appears. Expanding the IB Lagrangian to second order around
# the one-rule solution puts the threshold at beta_c = 1 / rho^2, where
# rho is the maximal correlation of X and Y, the second singular value of
# Q(x, y) = p(x, y) / sqrt(p(x) p(y)); rho^2 is also the slope of the
# bottleneck curve at the origin. Check: start two soft rules a hair
# away from the one-rule solution (1e-6) and see whether they split.
# Source: the illustrative guardrail source of p11.
import math

PX = [0.40, 0.18, 0.10, 0.07, 0.10, 0.06, 0.05, 0.04]
PYX = [[0.92, 0.06, 0.02], [0.25, 0.70, 0.05], [0.30, 0.55, 0.15],
       [0.10, 0.60, 0.30], [0.05, 0.15, 0.80], [0.03, 0.12, 0.85],
       [0.02, 0.08, 0.90], [0.01, 0.04, 0.95]]
PY = [math.fsum(PX[x] * PYX[x][y] for x in range(8)) for y in range(3)]


def rho_squared():
    q = [[PX[x] * PYX[x][y] / math.sqrt(PX[x] * PY[y]) for y in range(3)]
         for x in range(8)]
    m = [[math.fsum(q[x][i] * q[x][j] for x in range(8)) for j in range(3)]
         for i in range(3)]
    top = [math.sqrt(p) for p in PY]
    m = [[m[i][j] - top[i] * top[j] for j in range(3)] for i in range(3)]
    v = [1.0, -0.5, 0.25]
    lam = 0.0
    for _ in range(500):
        w = [math.fsum(m[i][j] * v[j] for j in range(3)) for i in range(3)]
        lam = math.sqrt(math.fsum(a * a for a in w))
        v = [a / lam for a in w]
    return lam


def kl(p, q):
    return math.fsum(a * math.log(a / b) for a, b in zip(p, q) if a > 0)


def run(beta, q, iters=2000):
    for _ in range(iters):
        pt = [math.fsum(PX[x] * q[x][t] for x in range(8)) for t in range(2)]
        pyt = [[math.fsum(PX[x] * q[x][t] * PYX[x][y] for x in range(8))
                / pt[t] for y in range(3)] for t in range(2)]
        new = []
        for x in range(8):
            lg = [math.log(pt[t]) - beta * kl(PYX[x], pyt[t])
                  for t in range(2)]
            m = max(lg)
            w = [math.exp(v - m) for v in lg]
            new.append([v / math.fsum(w) for v in w])
        q = new
    return q


def info(q):
    pt = [math.fsum(PX[x] * q[x][t] for x in range(8)) for t in range(2)]
    ixt = math.fsum(PX[x] * q[x][t] * math.log2(q[x][t] / pt[t])
                    for x in range(8) for t in range(2))
    pty = [[math.fsum(PX[x] * q[x][t] * PYX[x][y] for x in range(8))
            for y in range(3)] for t in range(2)]
    ity = math.fsum(pty[t][y] * math.log2(pty[t][y] / (pt[t] * PY[y]))
                    for t in range(2) for y in range(3))
    return max(ixt, 0.0), max(ity, 0.0)


r2 = rho_squared()
bc = 1.0 / r2
print("maximal correlation squared rho^2 = %.4f" % r2)
print("predicted critical beta 1/rho^2  = %.4f" % bc)
print("  beta/beta_c   beta    I(X;T)    I(T;Y)   I(T;Y)/I(X;T)")
for f in [0.90, 0.97, 1.03, 1.10, 1.25, 1.50]:
    q = [[0.5 + 1e-6 * (x - 3.5), 0.5 - 1e-6 * (x - 3.5)]
         for x in range(8)]
    ixt, ity = info(run(f * bc, q, 6000))
    ratio = "     -" if ixt < 1e-7 else "%6.4f" % (ity / ixt)
    print("     %4.2f     %6.4f  %8.6f  %8.6f  %s"
          % (f, f * bc, ixt, ity, ratio))

Single source and citation

Fan, Y. (2026). The Information Bottleneck and Rate-Distortion Limits: How Far a Bilingual Monograph Can Be Compressed into Guardrails before Discriminative Power Falls Away (Version 1.0.0), Chapter 4: Phase Transitions: Maximal Correlation, the Critical Exchange Rate and Deterministic Annealing. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-250 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026250,
  author    = {Fan, Yafu},
  title     = {The Information Bottleneck and Rate-Distortion Limits: How Far a Bilingual Monograph Can Be Compressed into Guardrails before Discriminative Power Falls Away},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book