What does Distilling Books into Skill Packs conclude, in numbers, about “Method: Skill Packs, Four Retention Indicators, Evidence Grades and the Four Steps of Distillation”?
Answer: Chapter 1 establishes two counter-intuitive results: pasting a whole book into the prompt gives coverage 1.000 but effective retention 0.000; with twelve samples all passing, the 95% upper bound on the failure rate is still 0.2209, and bringing it below one percent takes 299; under stated assumptions, a twelve-rule pack without samples loses at least one rule within ten edits with probability 0.9115.
Scope of the numbers: All samples are invented; pricing and closing rules are written in general terms and are not legal or compliance advice.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.2209, 0.9115, 299, 95
| golden n all pass | 95% upper bound on failure rate 3/n |
| 12 | 0.2209 0.2500 |
| 24 | 0.1173 0.1250 |
| 50 | 0.0582 0.0600 |
| 100 | 0.0295 0.0300 |
| 299 | 0.0100 0.0100 |
| 300 | 0.0099 0.0100 |
| samples needed so that zero failures bound the rate below 0.05 | 59 |
| samples needed so that zero failures bound the rate below 0.01 | 299 |
| q | 0.02 per rule per round; share of rules intact after n rounds |
| coverage c n | 1 n=5 n=10 n=20 P(any of 12 broken by n=10) |
| 0.0 | 0.9800 0.9039 0.8171 0.6676 0.9115 |
| 0.5 | 0.9900 0.9510 0.9044 0.8179 0.7006 |
| 0.9 | 0.9980 0.9900 0.9802 0.9608 0.2136 |
| 1.0 | 1.0000 1.0000 1.0000 1.0000 0.0000 |
| with no tests, half of the rules are gone after 34.3 rounds |
Show the full program
# Chapter 1. p12: What "all golden samples pass" proves, and how fast an untested pack decays (closed form).
# Part 1, zero numerator: if n independent golden samples all pass, the one-sided 95% upper bound on
# the per-case failure rate is 1 - 0.05 ** (1 / n), close to 3 / n (the "rule of three").
# Part 2, silent decay (a model, parameters assumed, not measured): each edit round breaks each rule
# with probability q = 0.02, independently; a golden suite notices a broken rule with probability c
# and the edit is reverted. A rule survives one round with probability 1 - q * (1 - c), so after
# n rounds the expected share of rules intact is (1 - q * (1 - c)) ** n. For a 12-rule pack the chance
# that at least one rule is broken is 1 - (1 - q * (1 - c)) ** (12 * n). Chapter 6 simulates this.
from math import ceil, log
Q = 0.02
RULES = 12
def upper95(n):
return 1 - 0.05 ** (1 / n)
if __name__ == "__main__":
print(" golden n all pass -> 95% upper bound on failure rate 3/n")
for n in (12, 24, 50, 100, 299, 300):
print(" {:8d} {:44.4f} {:.4f}".format(n, upper95(n), 3 / n))
for target in (0.05, 0.01):
need = ceil(log(0.05) / log(1 - target))
print(" samples needed so that zero failures bound the rate below {:.2f}: {}".format(target, need))
print(" q = {:.2f} per rule per round; share of rules intact after n rounds".format(Q))
print(" coverage c n=1 n=5 n=10 n=20 P(any of 12 broken by n=10)")
for c in (0.0, 0.5, 0.9, 1.0):
s = 1 - Q * (1 - c)
row = ["{:7.4f}".format(s ** n) for n in (1, 5, 10, 20)]
print(" {:10.1f} {} {:25.4f}".format(c, " ".join(row), 1 - s ** (RULES * 10)))
half = log(0.5) / log(1 - Q)
print(" with no tests, half of the rules are gone after {:.1f} rounds".format(half))Single source and citation
Fan, Y. (2026). Lossless Distillation of Monographs into Agent Skill Packs, Executable Rule Compilation and Self-Evolving Systems (Version 0.1.0), Chapter 1: Method: Skill Packs, Four Retention Indicators, Evidence Grades and the Four Steps of Distillation. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-150 · v0.1.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026150,
author = {Fan, Yafu},
title = {Lossless Distillation of Monographs into Agent Skill Packs, Executable Rule Compilation and Self-Evolving Systems},
year = {2026},
publisher = {5D Library},
edition = {0.1.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 2 · Extraction: Picking the Executable Sentences out of a Manuscript One by One
- Chapter 3 · Compilation: Turning Rule Sentences into Predicates, Decision Tables and Decision Trees
- Chapter 4 · Guardrails and Validators: Banned Phrases, Required Sentences, Refusal Conditions, Lint and Golden Samples
- Chapter 5 · One Specification for the Four Kits: Pricing Calculator, Front-End Style Library, Persuasive-Copy Checker and Customer-Service Closing Pack
- Chapter 6 · Self-Evolution: Feedback Loops, Regression Gates, Drift and the Accumulation of a Skill Library
- Chapter 7 · Governance: Least Privilege, Machine-Checked Red Lines, Disclaimers, Provenance and Rule Retirement
- Chapter 8 · Practice: Distilling a Twelve-Rule Pricing Book into a Pricing Calculator