What does Hundred-Volume Hypergraphs and Knowledge Scaling conclude, in numbers, about “Compression and Coverage: How Much Do Fifteen Primitives Really Explain?”?

Answer: Chapter 3 finds that a shared library of primitives shortens a naive description 6.10 times but only 1.04 times against a good general compressor, and that fifteen core primitives cover 82 of the 100 volumes of a synthetic library, full coverage needing 28.

Scope of the numbers: Its synthetic library, rates and hazards are illustrative models; all calculations are generated by its programs; the book is not investment, legal, medical or technical advice. Chapter 3 finds that a shared library of primitives shortens a naive description 6.10 times but only 1.04 times against a good general compressor, and that fifteen core primitives cover 82 of the 100 volumes of a synthetic library, full coverage needing 28.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 1.04, 28, 6.10

description raw bytes bz2 bytes lzma bytes
separate124,389 20,291 13,864
library20,387 13,167 13,376
primitives used28 (15 core, 13 rare)
raw ratio separate / library6.10
lzma ratio separate / library1.04
ideal bits (5 per letter): separate 621,945, library 101,935
Show the full program
# Chapter 3. p06: How much does a shared library of primitives compress
# a hundred-volume description? Synthetic incidence (shared with p07 and
# p08): 100 volumes each use 2 to 5 of 15 core primitives (popularity
# falling as 1 / rank^0.8) and, with chance 0.25, one of 30 rare ones.
# Each primitive has a random definition of 300 to 580 letters from a
# 32-letter alphabet (5 bits a letter, incompressible by design); each
# use carries 24 letters of parameters. "Separate": every volume spells
# out each definition it uses. "Library": each used definition once,
# then a reference per use. Kolmogorov complexity cannot be computed; bz2
# and lzma sizes are upper bounds on it. (bz2 rather than zlib: zlib's
# output size varies between zlib builds.) Seed 100.
import bz2
import lzma
import random

ALPHA = "abcdefghijklmnopqrstuvwxyz012345"

def incidence(rng):
    w = [1 / (j + 1) ** 0.8 for j in range(15)]
    vols = []
    for _ in range(100):
        k = rng.choice((2, 2, 3, 3, 3, 4, 4, 5))
        use = set()
        while len(use) < k:
            use.add(rng.choices(range(15), weights=w)[0])
        use = sorted(use)
        if rng.random() < 0.25:
            use.append(15 + rng.randrange(30))
        vols.append(use)
    return vols

if __name__ == "__main__":
    rng = random.Random(100)
    vols = incidence(rng)
    defs = ["".join(rng.choice(ALPHA) for _ in range(300 + 7 * j % 281))
            for j in range(45)]
    used = sorted({j for v in vols for j in v})
    sep, ref = [], ["".join("{}:{};".format(j, defs[j]) for j in used)]
    for v in vols:
        for j in v:
            par = "".join(rng.choice(ALPHA) for _ in range(24))
            sep.append(defs[j] + par)
            ref.append("#{}{}".format(j, par))
    a, b = "".join(sep).encode(), "".join(ref).encode()
    print("   description      raw bytes    bz2 bytes   lzma bytes")
    for name, s in (("separate", a), ("library", b)):
        print("   {:<14}  {:10,d}  {:11,d}  {:11,d}".format(
            name, len(s), len(bz2.compress(s, 9)),
            len(lzma.compress(s, preset=9))))
    print("   primitives used: {} ({} core, {} rare)".format(
        len(used), sum(j < 15 for j in used), sum(j >= 15 for j in used)))
    print("   raw ratio separate / library: {:.2f}".format(len(a) / len(b)))
    print("   lzma ratio separate / library: {:.2f}".format(
        len(lzma.compress(a, preset=9)) / len(lzma.compress(b, preset=9))))
    print("   ideal bits (5 per letter): separate {:,}, library {:,}".format(
        5 * sum(len(x) for x in sep), 5 * sum(len(x) for x in ref)))

Single source and citation

Fan, Y. (2026). Hundred-Volume Knowledge Hypergraphs, Multimodal Personal Prior Compilation and Autonomous Empire Scaling (Version 1.0.0), Chapter 3: Compression and Coverage: How Much Do Fifteen Primitives Really Explain?. 5D Library. https://doi.org/10.5281/zenodo.24090260

DOI: 10.5281/zenodo.24090260 · Read the monograph · 5DL-B-100 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026100,
  author    = {Fan, Yafu},
  title     = {Hundred-Volume Knowledge Hypergraphs, Multimodal Personal Prior Compilation and Autonomous Empire Scaling},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.24090260},
  url       = {https://doi.org/10.5281/zenodo.24090260}
}

Other chapter answers in this book