What does Ockham's Razor for Rule Bases conclude, in numbers, about “Sources East and West: the Extra Finger of the Zhuangzi and Ockham's "In Vain through More"”?
Answer: "Lengthening the duck's legs" replaces a short rule by a longer version, which narrows a true rule. The rule half grows by only 2.0 bits, but the records narrowed away become exceptions again, the exception half grows by 45.6 bits, the total by 47.6 bits, and the hold-out error rises by 0.0307. "Cutting the crane's legs" shortens a long rule by removing a condition, which widens a true rule. The rule half actually saves 5.3 bits, but the widened rule wrongly acts on many records it should leave alone, the exception half grows by 152.5 bits, the total by 147.2 bits, and the hold-out error rises by 0.1408. "Amputation" deletes a true rule outright: the rules save 10.5 bits, the exceptions grow by 74.6 bits, and the hold-out error rises by 0.0507.
Scope of the numbers: All decision records are synthetic, all parameters illustrative, and all calculations generated by the book's programs; it is general business analysis, not personalised business advice. Marked boxes separate published evidence, runnable Python calculations (18 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.0307, 0.0507, 0.1408, 147.2, 152.5, 2.0
| base: L(H) 24.4, L(D|H) 407.6, hold-out 0.1973 | ||||
| edit | dL(H) | dL(D|H) | dtotal | d holdout |
| extra finger | 14.9 | 0.0 | 14.9 | +0.0000 |
| wart | 12.5 | 0.0 | 12.5 | +0.0000 |
| lengthen duck | 2.0 | 45.6 | 47.6 | +0.0307 |
| cut crane | -5.3 | 152.5 | 147.2 | +0.1408 |
| amputate | -10.5 | 74.6 | 64.2 | +0.0507 |
Show the full program
# Chapter 7. p71: the webbed toe, the extra finger, the duck's leg and
# the crane's leg, written as edits to a rule base and priced in bits.
# Base: the two true rules on the seeded 600-record ledger of p23,
# 4000 hold-out records. Edits (our reading of the Zhuangzi images):
# extra finger add a narrower copy of a true rule (one condition more)
# wart add a verbatim second copy of a true rule
# lengthen duck replace a true rule by a narrower version
# cut crane replace a true rule by a broader version
# amputate delete a true rule
# Reported: change in L(H), in L(D|H), in the total, and in hold-out
# error. A positive total means the edit makes the base worse.
import itertools
import math
import random
V = (4, 4, 3, 4, 3, 5)
A = len(V)
NAMES = ("ch", "bd", "sg", "ss", "sz", "rg")
def truth(x):
if x[0] == 0 and x[1] == 3:
return 0.85
if x[2] == 2 and x[3] == 1:
return 0.80
return 0.45 if x[5] == 4 else 0.12
def records(rng, n):
out = []
for _ in range(n):
x = tuple(rng.randrange(v) for v in V)
out.append((x, 1 if rng.random() < truth(x) else 0))
return out
def gamma(n):
return 2 * (n.bit_length() - 1) + 1
def rule_bits(r):
m = len(r)
t = [float(gamma(m)), math.log2(math.comb(A, m))]
t += [math.log2(V[a]) for a, _ in r]
return math.fsum(t)
def exc_bits(n, e):
return gamma(e + 1) + math.log2(math.comb(n, e))
def candidates():
out = []
for m in (1, 2, 3):
for at in itertools.combinations(range(A), m):
for vs in itertools.product(*[range(V[a]) for a in at]):
out.append(tuple(zip(at, vs)))
return out
def cover_masks(data, cands):
am, pos = {}, 0
for i, (x, y) in enumerate(data):
for a in range(A):
am[(a, x[a])] = am.get((a, x[a]), 0) | (1 << i)
pos |= y << i
full = (1 << len(data)) - 1
cov = []
for r in cands:
c = full
for cond in r:
c &= am.get(cond, 0)
cov.append(c)
return cov, pos, full
def show(r):
return "&".join("{}={}".format(NAMES[a], v) for a, v in r)
def greedy_path(cov, pos, full, kmax=40):
unc, path = full, []
while len(path) < kmax:
best, bi = 0, -1
for i, c in enumerate(cov):
nw = c & unc
g = (nw & pos).bit_count() - (nw & ~pos).bit_count()
if g > best:
best, bi = g, i
if bi < 0:
break
path.append((bi, best))
unc &= ~cov[bi]
return path
def errors(rules, cov, pos, full):
act = 0
for i in rules:
act |= cov[i]
return (pos & ~act & full).bit_count() + (act & ~pos).bit_count()
def ledger(rules, cands, cov, pos, full, n):
e = errors([index_of(cands, r) for r in rules], cov, pos, full)
lh = math.fsum([float(gamma(len(rules) + 1))]
+ [rule_bits(r) for r in rules])
return lh, exc_bits(n, e)
def index_of(cands, r):
return cands.index(tuple(sorted(r)))
if __name__ == "__main__":
rng = random.Random(20261113)
tr, te = records(rng, 600), records(rng, 4000)
cands = candidates()
cov, pos, full = cover_masks(tr, cands)
tcov, tpos, tfull = cover_masks(te, cands)
r1, r2 = ((0, 0), (1, 3)), ((2, 2), (3, 1))
base = [r1, r2]
edits = [("extra finger", [r1, r2, ((0, 0), (1, 3), (4, 1))]),
("wart", [r1, r2, r2]),
("lengthen duck", [((0, 0), (1, 3), (4, 1)), r2]),
("cut crane", [r1, ((2, 2),)]),
("amputate", [r1])]
lh0, ld0 = ledger(base, cands, cov, pos, full, 600)
ids = lambda rs: [index_of(cands, r) for r in rs]
h0 = errors(ids(base), tcov, tpos, tfull) / 4000
print("base: L(H) {:.1f}, L(D|H) {:.1f}, hold-out {:.4f}".format(
lh0, ld0, h0))
print(" edit dL(H) dL(D|H) dtotal d holdout")
for name, rs in edits:
lh, ld = ledger(rs, cands, cov, pos, full, 600)
h = errors(ids(rs), tcov, tpos, tfull) / 4000
print(" {:<14} {:>6.1f} {:>7.1f} {:>6.1f} {:>+9.4f}".format(
name, lh - lh0, ld - ld0, lh + ld - lh0 - ld0, h - h0))Single source and citation
Fan, Y. (2026). The Minimum Description Length Principle and Ockham's Razor for Rule Bases: Two-Part Codes that Decide Whether a New Rule Earns Its Place (Version 1.0.0), Chapter 7: Sources East and West: the Extra Finger of the Zhuangzi and Ockham's "In Vain through More". 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-311 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026311,
author = {Fan, Yafu},
title = {The Minimum Description Length Principle and Ockham's Razor for Rule Bases: Two-Part Codes that Decide Whether a New Rule Earns Its Place},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Even Integers Cost Something: Universal Codes, the Kraft Inequality and the Bits of an Exception List
- Chapter 2 · The Two-Part Ledger: How Many Exceptions a Rule Must Fix to Earn Its Own Length
- Chapter 3 · The Overfitting Curve: Rule Count, Total Description Length and Hold-out Error
- Chapter 4 · Reconciling with the Bayesian Information Criterion: Parametric Complexity, Normalised Maximum Likelihood and Where Rule Bases Part Ways
- Chapter 5 · Sensitivity to the Code: Six Reasonable Codes, Six Verdicts, and How Many Bits of Slack to Reserve
- Chapter 6 · The Pruner's Admission Threshold: a Gate that Only Catches Repeats and a Gate that Counts Bits
- Chapter 8 · Closing the Loop: the Admission Check-up, the Rule Admission Description-Length Referee and a Pre-registered Plan of Refutation