What does Distilling Books into Skill Packs conclude, in numbers, about “Practice: Distilling a Twelve-Rule Pricing Book into a Pricing Calculator”?

Answer: Chapter 8 distils an invented twelve-rule pricing book into a pricing calculator: a signal-word scan reaches precision 0.714 and recall 0.833 and misses exactly the definition of margin and one refusal condition; one casual edit drops the pass rate to 0.788 and refusal completeness to 0.833 while coverage and executable rate stay at 1; the pre-registered three-arm comparison is set at 228 questions per arm, 684 answers in all, with an upper bound of 0.0131 if no violation is seen.

Scope of the numbers: All samples are invented; pricing and closing rules are written in general terms and are not legal or compliance advice.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.0131, 228, 684

contrastratesalphapowern/armn/arm (x12)per rule
A vs C0.20 vs 0.050.0250.8092968
A vs C0.20 vs 0.050.0250.9011912010
B vs C0.15 vs 0.050.0250.8017018015
B vs C0.15 vs 0.050.0250.9022222819
A vs B0.20 vs 0.150.0500.8090691276
A vs B0.20 vs 0.150.0500.9012121212101
registered size: 228 questions per arm, 684 answers in total over 3 arms
if arm C shows 0 violations in 228: 95% upper bound 0.0131
questions needed for a zero-violation claim below 0.01: 299
Show the full program
# Chapter 8. p83: Sample sizes for the pre-registered comparison of three ways to hand the book to an agent.
# Arms: A the whole book pasted as a long prompt; B the instruction file only (rules as prose);
# C the full pack (instruction file + validators + decision table + golden samples gating every answer).
# Primary outcome: share of answers to held-out pricing questions that break at least one of the 12
# rules, judged by the validators and by a blind reviewer. Planning rates are assumptions, not data:
# pA = 0.20, pB = 0.15, pC = 0.05. Two primary contrasts (A vs C, B vs C), each at two-sided alpha
# 0.025 (Bonferroni for 0.05 overall). n per arm for two independent proportions:
#   n = (z_a * sqrt(2 pbar qbar) + z_b * sqrt(pA qA + pC qC)) ** 2 / (pA - pC) ** 2
# Questions are drawn in equal numbers for each of the 12 rules, so n is rounded up to a multiple of 12.
from math import ceil, log, sqrt
from statistics import NormalDist

PLAN = {"A": 0.20, "B": 0.15, "C": 0.05}
RULES = 12


def n_two_prop(p1, p2, alpha, power):
    za = NormalDist().inv_cdf(1 - alpha / 2)
    zb = NormalDist().inv_cdf(power)
    pbar = (p1 + p2) / 2
    num = za * sqrt(2 * pbar * (1 - pbar)) + zb * sqrt(p1 * (1 - p1) + p2 * (1 - p2))
    return ceil(num ** 2 / (p1 - p2) ** 2)


def to_rules(n):
    return RULES * ceil(n / RULES)


if __name__ == "__main__":
    print("   contrast  rates          alpha  power  n/arm  n/arm (x12)  per rule")
    for a, b, alpha in (("A", "C", 0.025), ("B", "C", 0.025), ("A", "B", 0.05)):
        for power in (0.80, 0.90):
            n = n_two_prop(PLAN[a], PLAN[b], alpha, power)
            print("   {} vs {}    {:.2f} vs {:.2f}   {:.3f}  {:.2f} {:6d} {:12d} {:9d}".format(
                a, b, PLAN[a], PLAN[b], alpha, power, n, to_rules(n), to_rules(n) // RULES))
    n = max(to_rules(n_two_prop(PLAN[x], PLAN["C"], 0.025, 0.90)) for x in "AB")
    print("   registered size: {} questions per arm, {} answers in total over 3 arms".format(n, 3 * n))
    print("   if arm C shows 0 violations in {}: 95% upper bound {:.4f}".format(n, 1 - 0.05 ** (1 / n)))
    need = ceil(log(0.05) / log(0.99))
    print("   questions needed for a zero-violation claim below 0.01: {}".format(need))

Single source and citation

Fan, Y. (2026). Lossless Distillation of Monographs into Agent Skill Packs, Executable Rule Compilation and Self-Evolving Systems (Version 0.1.0), Chapter 8: Practice: Distilling a Twelve-Rule Pricing Book into a Pricing Calculator. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-150 · v0.1.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026150,
  author    = {Fan, Yafu},
  title     = {Lossless Distillation of Monographs into Agent Skill Packs, Executable Rule Compilation and Self-Evolving Systems},
  year      = {2026},
  publisher = {5D Library},
  edition   = {0.1.0},
  note      = {DOI pending}
}

Other chapter answers in this book