What does Ockham's Razor for Rule Bases conclude, in numbers, about “Even Integers Cost Something: Universal Codes, the Kraft Inequality and the Bits of an Exception List”?
Answer: Program p13 compares the four at two scales, 600 records and 6,000. With 600 records and 30 exceptions, flagging costs 600 bits, listing positions 309, and the enumerative code only 177.1, just 5.25 bits above Shannon's limit of 171.8. Per exception, the enumerative code spends 5.90 bits and position listing 10.30. With 6,000 records and 600 exceptions, listing positions takes 7,819 bits, the enumerative code 2,827.1 and the limit is 2,814.0.
Scope of the numbers: All decision records are synthetic, all parameters illustrative, and all calculations generated by the book's programs; it is general business analysis, not personalised business advice. Marked boxes separate published evidence, runnable Python calculations (18 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 10.30, 171.8, 177.1, 2814.0, 2827.1, 30
| bits to name the exceptions among N records | |||||
| N | e | flags | index | enum | bound |
| 600 | 6 | 600 | 65 | 50.8 | 48.5 |
| 600 | 30 | 600 | 309 | 177.1 | 171.8 |
| 600 | 60 | 600 | 611 | 288.2 | 281.4 |
| 600 | 120 | 600 | 1213 | 441.5 | 433.2 |
| 600 | 180 | 600 | 1815 | 539.0 | 528.8 |
| 600 | 300 | 600 | 3017 | 612.1 | 600.0 |
| 6000 | 60 | 6000 | 791 | 491.5 | 484.8 |
| 6000 | 300 | 6000 | 3917 | 1730.0 | 1718.4 |
| 6000 | 600 | 6000 | 7819 | 2827.1 | 2814.0 |
| 6000 | 1200 | 6000 | 15621 | 4346.3 | 4331.6 |
| 6000 | 1800 | 6000 | 23421 | 5302.3 | 5287.7 |
| 6000 | 3000 | 6000 | 39023 | 6016.4 | 6000.0 |
| N 600, e 30: enum per exception 5.90 bits, index 10.30 | |||||
| enum minus bound at N 600, e 30: 5.25 bits |
Show the full program
# Chapter 1. p13: bits to say WHICH of N decision records break the
# rules (the exceptions), four ways:
# flags one bit per record N
# index gamma(e+1) + e * ceil(log2 N) (list the positions)
# enum gamma(e+1) + log2 C(N, e) (count, then which subset)
# bound N h(e/N), the Shannon entropy limit (not a real code)
# N = 600 and 6000 records; e = number of exceptions. Exact arithmetic.
import math
def gamma(n):
return 2 * (n.bit_length() - 1) + 1
def h2(p):
if p <= 0 or p >= 1:
return 0.0
return -p * math.log2(p) - (1 - p) * math.log2(1 - p)
def costs(n, e):
index = gamma(e + 1) + e * math.ceil(math.log2(n))
enum = gamma(e + 1) + math.log2(math.comb(n, e))
return n, index, enum, n * h2(e / n)
if __name__ == "__main__":
print("bits to name the exceptions among N records")
print(" N e flags index enum bound")
for n in (600, 6000):
for share in (0.01, 0.05, 0.10, 0.20, 0.30, 0.50):
e = round(n * share)
f, i, en, b = costs(n, e)
print(" {:>4} {:>4} {:>6} {:>7} {:>7.1f} {:>7.1f}"
.format(n, e, f, i, en, b))
n, e = 600, 30
f, i, en, b = costs(n, e)
print("N 600, e 30: enum per exception {:.2f} bits,"
" index {:.2f}".format(en / e, i / e))
print("enum minus bound at N 600, e 30: {:.2f} bits".format(en - b))Single source and citation
Fan, Y. (2026). The Minimum Description Length Principle and Ockham's Razor for Rule Bases: Two-Part Codes that Decide Whether a New Rule Earns Its Place (Version 1.0.0), Chapter 1: Even Integers Cost Something: Universal Codes, the Kraft Inequality and the Bits of an Exception List. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-311 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026311,
author = {Fan, Yafu},
title = {The Minimum Description Length Principle and Ockham's Razor for Rule Bases: Two-Part Codes that Decide Whether a New Rule Earns Its Place},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 2 · The Two-Part Ledger: How Many Exceptions a Rule Must Fix to Earn Its Own Length
- Chapter 3 · The Overfitting Curve: Rule Count, Total Description Length and Hold-out Error
- Chapter 4 · Reconciling with the Bayesian Information Criterion: Parametric Complexity, Normalised Maximum Likelihood and Where Rule Bases Part Ways
- Chapter 5 · Sensitivity to the Code: Six Reasonable Codes, Six Verdicts, and How Many Bits of Slack to Reserve
- Chapter 6 · The Pruner's Admission Threshold: a Gate that Only Catches Repeats and a Gate that Counts Bits
- Chapter 7 · Sources East and West: the Extra Finger of the Zhuangzi and Ockham's "In Vain through More"
- Chapter 8 · Closing the Loop: the Admission Check-up, the Rule Admission Description-Length Referee and a Pre-registered Plan of Refutation