What does The Limits of Guardrail Compression conclude, in numbers, about “Blahut and Arimoto: Computing the Rate-Distortion Function of a Guardrail in the Standard Library”?

Answer: Chapter 3 implements Blahut and Arimoto in the standard library, matches the binary closed form digit by digit, and shows that draft-by-draft rules use 0.2620 bits more than the rate-distortion curve.

Scope of the numbers: Chapter 1 builds an illustrative source of eight kinds of draft and three verdicts and the counting bound: three rules keep 0.960 of the verdict information, two only 0.629. Chapter 4 shows that the critical exchange rate is the reciprocal of the squared maximal correlation, 0.6844 and 1.4610 on the illustrative source while information rises continuously as rules split.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.2620

expected cost: one action for all 1.5262, full detail 0.4144
slope D (cost) R(D) bits rules >2^R
-0.31.2776 0.0630 1.04
-0.61.0403 0.2138 1.16
-1.00.7505 0.5437 1.46
-1.50.6050 0.7981 1.74
-2.50.4966 1.0971 2.14
-4.00.4427 1.3384 2.53
-8.00.4170 1.5312 2.89
best per-draft rule maps vs the curve at the same cost:
k cost H(A) bits R(cost) excess
11.5262 0.0000 0.0000 0.0000
20.6494 0.9710 0.7090 0.2620
30.4144 1.5690 1.5690 0.0000
Show the full program
# Chapter 3. p32: the rate-distortion curve of a guardrail. Source = the
# eight draft kinds of p11; reproduction = the gate's action (pass,
# revise, block); distortion d(x, a) = expected cost of action a on kind
# x under an illustrative cost table (passing a draft that should be
# blocked costs 10). Blahut-Arimoto gives R(D); a rule set of k distinct
# outputs carries at most log2 k bits, so it needs k >= 2 ** R(D).
# Last part: the best one-draft-at-a-time rule maps (all 3 ** 8) sit
# above the curve; Shannon's R(D) is reached only by coding long blocks.
import itertools
import math

PX = [0.40, 0.18, 0.10, 0.07, 0.10, 0.06, 0.05, 0.04]
PYX = [[0.92, 0.06, 0.02], [0.25, 0.70, 0.05], [0.30, 0.55, 0.15],
       [0.10, 0.60, 0.30], [0.05, 0.15, 0.80], [0.03, 0.12, 0.85],
       [0.02, 0.08, 0.90], [0.01, 0.04, 0.95]]
COST = [[0.0, 1.0, 3.0], [2.0, 0.0, 1.0], [10.0, 4.0, 0.0]]
DIST = [[math.fsum(PYX[x][y] * COST[y][a] for y in range(3))
         for a in range(3)] for x in range(8)]


def blahut(s, iters=50000, tol=1e-14):
    r = [1.0 / 3] * 3
    for _ in range(iters):
        q = []
        for x in range(8):
            w = [r[a] * math.exp(s * DIST[x][a]) for a in range(3)]
            tot = math.fsum(w)
            q.append([v / tot for v in w])
        new = [math.fsum(PX[x] * q[x][a] for x in range(8))
               for a in range(3)]
        gap = max(abs(u - v) for u, v in zip(new, r))
        r = new
        if gap < tol:
            break
    d = math.fsum(PX[x] * q[x][a] * DIST[x][a]
                  for x in range(8) for a in range(3))
    rate = math.fsum(PX[x] * q[x][a] * math.log2(q[x][a] / r[a])
                     for x in range(8) for a in range(3) if q[x][a] > 0)
    return d, max(rate, 0.0)


def rate_at(target):
    lo, hi = -60.0, 0.0
    for _ in range(60):
        mid = (lo + hi) / 2
        if blahut(mid)[0] > target:
            hi = mid
        else:
            lo = mid
    return blahut(lo)[1]


dmax = min(math.fsum(PX[x] * DIST[x][a] for x in range(8))
           for a in range(3))
dmin = math.fsum(PX[x] * min(DIST[x]) for x in range(8))
print("expected cost: one action for all %.4f, full detail %.4f"
      % (dmax, dmin))
print("  slope   D (cost)  R(D) bits  rules >= 2^R")
for s in [-0.3, -0.6, -1.0, -1.5, -2.5, -4.0, -8.0]:
    d, rate = blahut(s)
    print("  %5.1f    %6.4f    %6.4f     %5.2f" % (s, d, rate, 2 ** rate))
best = {}
for m in itertools.product(range(3), repeat=8):
    k = len(set(m))
    d = math.fsum(PX[x] * DIST[x][m[x]] for x in range(8))
    if k not in best or d < best[k][0] - 1e-12:
        best[k] = (d, m)
print("best per-draft rule maps vs the curve at the same cost:")
print("  k  cost    H(A) bits  R(cost)  excess")
for k in sorted(best):
    d, m = best[k]
    pa = [math.fsum(PX[x] for x in range(8) if m[x] == a) for a in range(3)]
    ha = 0.0 - math.fsum(p * math.log2(p) for p in pa if p > 0)
    rd = rate_at(d) if d < dmax - 1e-9 else 0.0
    print("  %d  %6.4f  %6.4f     %6.4f   %6.4f" % (k, d, ha, rd, ha - rd))

Single source and citation

Fan, Y. (2026). The Information Bottleneck and Rate-Distortion Limits: How Far a Bilingual Monograph Can Be Compressed into Guardrails before Discriminative Power Falls Away (Version 1.0.0), Chapter 3: Blahut and Arimoto: Computing the Rate-Distortion Function of a Guardrail in the Standard Library. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-250 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026250,
  author    = {Fan, Yafu},
  title     = {The Information Bottleneck and Rate-Distortion Limits: How Far a Bilingual Monograph Can Be Compressed into Guardrails before Discriminative Power Falls Away},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book