What does Ockham's Razor for Rule Bases conclude, in numbers, about “The Two-Part Ledger: How Many Exceptions a Rule Must Fix to Earn Its Own Length”?

Answer: Chapter 2 sets up the two-part ledger: a narrow rule that is right more than eight times in ten but fixes only 7 exceptions net cannot earn its own length; on a synthetic ledger the gate admits only 3 of the proposer's 12 rules, the third with a margin of just 2.9 bits.

Scope of the numbers: All decision records are synthetic, all parameters illustrative, and all calculations generated by the book's programs; it is general business analysis, not personalised business advice. Marked boxes separate published evidence, runnable Python calculations (18 programs), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 12, 2.9

1347 candidates, N 600, exceptions with no rule 172
kproposed rulenewnetsavecostmargingate
1sg=2&ss=1553757.012.544.5ADMIT
2ch=0&bd=3373164.210.953.3ADMIT
3ss=3&rg=423716.213.22.9ADMIT
4ch=3&rg=41937.211.2-4.0reject
5sg=1&ss=2&rg=45512.313.2-0.9reject
6ch=1&bd=3&rg=4825.113.6-8.6reject
7bd=1&ss=0&rg=4225.115.6-10.5reject
8bd=0&sz=1&rg=1625.213.2-8.0reject
9bd=1&sz=0&rg=4225.313.2-7.9reject
10ss=2&sz=1&rg=1425.413.2-7.8reject
11ch=0&sz=2&rg=2312.713.2-10.5reject
12bd=3&sz=0&rg=4112.713.2-10.5reject
proposer offered 12, gate admitted 3
Show the full program
# Chapter 2. p23: the MDL admission gate on synthetic decision records.
# Records (seeded, illustrative): six attributes ch bd sg ss sz rg with
# V = (4, 4, 3, 4, 3, 5) values; act = 1 with probability 0.85 if
# ch=0 and bd=3, 0.80 if sg=2 and ss=1, 0.45 if rg=4, else 0.12.
# Candidates: every conjunction of 1 to 3 conditions, THEN act.
# A greedy proposer offers, step by step, the candidate that removes the
# most training exceptions; the gate admits while margin > 0 (p21 code).
import itertools
import math
import random

V = (4, 4, 3, 4, 3, 5)
A = len(V)
NAMES = ("ch", "bd", "sg", "ss", "sz", "rg")


def truth(x):
    if x[0] == 0 and x[1] == 3:
        return 0.85
    if x[2] == 2 and x[3] == 1:
        return 0.80
    return 0.45 if x[5] == 4 else 0.12


def records(rng, n):
    out = []
    for _ in range(n):
        x = tuple(rng.randrange(v) for v in V)
        out.append((x, 1 if rng.random() < truth(x) else 0))
    return out


def gamma(n):
    return 2 * (n.bit_length() - 1) + 1


def rule_bits(r):
    m = len(r)
    t = [float(gamma(m)), math.log2(math.comb(A, m))]
    t += [math.log2(V[a]) for a, _ in r]
    return math.fsum(t)


def exc_bits(n, e):
    return gamma(e + 1) + math.log2(math.comb(n, e))


def candidates():
    out = []
    for m in (1, 2, 3):
        for at in itertools.combinations(range(A), m):
            for vs in itertools.product(*[range(V[a]) for a in at]):
                out.append(tuple(zip(at, vs)))
    return out


def cover_masks(data, cands):
    am, pos = {}, 0
    for i, (x, y) in enumerate(data):
        for a in range(A):
            am[(a, x[a])] = am.get((a, x[a]), 0) | (1 << i)
        pos |= y << i
    full = (1 << len(data)) - 1
    cov = []
    for r in cands:
        c = full
        for cond in r:
            c &= am.get(cond, 0)
        cov.append(c)
    return cov, pos, full


def show(r):
    return "&".join("{}={}".format(NAMES[a], v) for a, v in r)


if __name__ == "__main__":
    rng = random.Random(20261113)
    tr = records(rng, 600)
    cands = candidates()
    cov, pos, full = cover_masks(tr, cands)
    n, unc, k = len(tr), full, 0
    e = (pos & full).bit_count()
    print("{} candidates, N {}, exceptions with no rule {}".format(
        len(cands), n, e))
    print("  k  proposed rule      new  net  save   cost  margin  gate")
    admitted = 0
    while True:
        best, bi = 0, -1
        for i, c in enumerate(cov):
            nw = c & unc
            g = (nw & pos).bit_count() - (nw & ~pos).bit_count()
            if g > best:
                best, bi = g, i
        if bi < 0:
            break
        nw = cov[bi] & unc
        save = exc_bits(n, e) - exc_bits(n, e - best)
        cost = rule_bits(cands[bi]) + gamma(k + 2) - gamma(k + 1)
        ok = save > cost and admitted == k
        admitted += 1 if ok else 0
        k, e, unc = k + 1, e - best, unc & ~cov[bi]
        print(" {:>2}  {:<17} {:>4} {:>4} {:>5.1f} {:>6.1f} {:>7.1f}  {}"
              .format(k, show(cands[bi]), nw.bit_count(), best, save,
                      cost, save - cost, "ADMIT" if ok else "reject"))
    print("proposer offered {}, gate admitted {}".format(k, admitted))

Single source and citation

Fan, Y. (2026). The Minimum Description Length Principle and Ockham's Razor for Rule Bases: Two-Part Codes that Decide Whether a New Rule Earns Its Place (Version 1.0.0), Chapter 2: The Two-Part Ledger: How Many Exceptions a Rule Must Fix to Earn Its Own Length. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-311 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026311,
  author    = {Fan, Yafu},
  title     = {The Minimum Description Length Principle and Ockham's Razor for Rule Bases: Two-Part Codes that Decide Whether a New Rule Earns Its Place},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book