What does The Limits of Guardrail Compression conclude, in numbers, about “Blahut and Arimoto: Computing the Rate-Distortion Function of a Guardrail in the Standard Library”?
Answer: Chapter 3 implements Blahut and Arimoto in the standard library, matches the binary closed form digit by digit, and shows that draft-by-draft rules use 0.2620 bits more than the rate-distortion curve.
Scope of the numbers: Chapter 1 builds an illustrative source of eight kinds of draft and three verdicts and the counting bound: three rules keep 0.960 of the verdict information, two only 0.629. Chapter 4 shows that the critical exchange rate is the reciprocal of the squared maximal correlation, 0.6844 and 1.4610 on the illustrative source while information rises continuously as rules split.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.2620
| expected cost: one action for all 1.5262, full detail 0.4144 | |
| slope D (cost) R(D) bits rules > | 2^R |
| -0.3 | 1.2776 0.0630 1.04 |
| -0.6 | 1.0403 0.2138 1.16 |
| -1.0 | 0.7505 0.5437 1.46 |
| -1.5 | 0.6050 0.7981 1.74 |
| -2.5 | 0.4966 1.0971 2.14 |
| -4.0 | 0.4427 1.3384 2.53 |
| -8.0 | 0.4170 1.5312 2.89 |
| best per-draft rule maps vs the curve at the same cost: | |
| k cost H(A) bits R(cost) excess | |
| 1 | 1.5262 0.0000 0.0000 0.0000 |
| 2 | 0.6494 0.9710 0.7090 0.2620 |
| 3 | 0.4144 1.5690 1.5690 0.0000 |
Show the full program
# Chapter 3. p32: the rate-distortion curve of a guardrail. Source = the
# eight draft kinds of p11; reproduction = the gate's action (pass,
# revise, block); distortion d(x, a) = expected cost of action a on kind
# x under an illustrative cost table (passing a draft that should be
# blocked costs 10). Blahut-Arimoto gives R(D); a rule set of k distinct
# outputs carries at most log2 k bits, so it needs k >= 2 ** R(D).
# Last part: the best one-draft-at-a-time rule maps (all 3 ** 8) sit
# above the curve; Shannon's R(D) is reached only by coding long blocks.
import itertools
import math
PX = [0.40, 0.18, 0.10, 0.07, 0.10, 0.06, 0.05, 0.04]
PYX = [[0.92, 0.06, 0.02], [0.25, 0.70, 0.05], [0.30, 0.55, 0.15],
[0.10, 0.60, 0.30], [0.05, 0.15, 0.80], [0.03, 0.12, 0.85],
[0.02, 0.08, 0.90], [0.01, 0.04, 0.95]]
COST = [[0.0, 1.0, 3.0], [2.0, 0.0, 1.0], [10.0, 4.0, 0.0]]
DIST = [[math.fsum(PYX[x][y] * COST[y][a] for y in range(3))
for a in range(3)] for x in range(8)]
def blahut(s, iters=50000, tol=1e-14):
r = [1.0 / 3] * 3
for _ in range(iters):
q = []
for x in range(8):
w = [r[a] * math.exp(s * DIST[x][a]) for a in range(3)]
tot = math.fsum(w)
q.append([v / tot for v in w])
new = [math.fsum(PX[x] * q[x][a] for x in range(8))
for a in range(3)]
gap = max(abs(u - v) for u, v in zip(new, r))
r = new
if gap < tol:
break
d = math.fsum(PX[x] * q[x][a] * DIST[x][a]
for x in range(8) for a in range(3))
rate = math.fsum(PX[x] * q[x][a] * math.log2(q[x][a] / r[a])
for x in range(8) for a in range(3) if q[x][a] > 0)
return d, max(rate, 0.0)
def rate_at(target):
lo, hi = -60.0, 0.0
for _ in range(60):
mid = (lo + hi) / 2
if blahut(mid)[0] > target:
hi = mid
else:
lo = mid
return blahut(lo)[1]
dmax = min(math.fsum(PX[x] * DIST[x][a] for x in range(8))
for a in range(3))
dmin = math.fsum(PX[x] * min(DIST[x]) for x in range(8))
print("expected cost: one action for all %.4f, full detail %.4f"
% (dmax, dmin))
print(" slope D (cost) R(D) bits rules >= 2^R")
for s in [-0.3, -0.6, -1.0, -1.5, -2.5, -4.0, -8.0]:
d, rate = blahut(s)
print(" %5.1f %6.4f %6.4f %5.2f" % (s, d, rate, 2 ** rate))
best = {}
for m in itertools.product(range(3), repeat=8):
k = len(set(m))
d = math.fsum(PX[x] * DIST[x][m[x]] for x in range(8))
if k not in best or d < best[k][0] - 1e-12:
best[k] = (d, m)
print("best per-draft rule maps vs the curve at the same cost:")
print(" k cost H(A) bits R(cost) excess")
for k in sorted(best):
d, m = best[k]
pa = [math.fsum(PX[x] for x in range(8) if m[x] == a) for a in range(3)]
ha = 0.0 - math.fsum(p * math.log2(p) for p in pa if p > 0)
rd = rate_at(d) if d < dmax - 1e-9 else 0.0
print(" %d %6.4f %6.4f %6.4f %6.4f" % (k, d, ha, rd, ha - rd))Single source and citation
Fan, Y. (2026). The Information Bottleneck and Rate-Distortion Limits: How Far a Bilingual Monograph Can Be Compressed into Guardrails before Discriminative Power Falls Away (Version 1.0.0), Chapter 3: Blahut and Arimoto: Computing the Rate-Distortion Function of a Guardrail in the Standard Library. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-250 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026250,
author = {Fan, Yafu},
title = {The Information Bottleneck and Rate-Distortion Limits: How Far a Bilingual Monograph Can Be Compressed into Guardrails before Discriminative Power Falls Away},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Guardrail Source: Eight Kinds of Draft, Three Verdicts and the Counting Bound on Rules
- Chapter 2 · The Axis: Cliffs, Concave Curves and the Straight Line under Deterministic Labels
- Chapter 4 · Phase Transitions: Maximal Correlation, the Critical Exchange Rate and Deterministic Annealing
- Chapter 5 · From Monograph to Guardrails: Agglomerative Bottlenecks, Five Knee Criteria and Bilingual Redundancy
- Chapter 6 · Opposing Evidence: Finite Samples, Learnt Compression and How Discriminative Power Is Scored
- Chapter 7 · Classical Sources East and West: "Writing Does Not Exhaust Speech, Speech Does Not Exhaust Intent" and Shannon's Suspension of Meaning
- Chapter 8 · Closing the Loop: One Compression-Recall Curve per Volume, a Knee Calculator and a Pre-Registered Protocol