What does Distilling Books into Skill Packs conclude, in numbers, about “Guardrails and Validators: Banned Phrases, Required Sentences, Refusal Conditions, Lint and Golden Samples”?
Answer: Guardrails come in three sentence types, banned phrases, required sentences and refusal conditions, each paired with replacement wording, trigger predicates and fixed replies; outside text is always data. Lint has a word half and a structural half. On 80 drafts the best word matcher reaches precision 0.897 and recall 0.963, while the structural check is exact when fields are complete, so any rule that can be expressed as fields is not checked with a word list. The unit costs of false alarms and misses set the threshold, which equals \(C_{FP}/(C_{FP}+C_{FN})\) and must be recomputed whenever the base rate moves; otherwise every thousand drafts pay the gap between 6231.4 and 4404.2. Golden samples belong on the boundaries: 18 boundary samples reach a mutation score of 1.000 where 200 random samples reach only 0.767. A clean run only gives a bound, and only 299 clean samples justify saying the error rate is below 0.01. The next chapter builds these checks into the unified specification of the four kits.
Scope of the numbers: All samples are invented; pricing and closing rules are written in general terms and are not legal or compliance advice.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.897, 0.963, 1.000, 80
| drafts 80, true violations 27, clean or benign 53 | |
| matcher TP FP FN TN precision recall F1 | |
| L0 | 12 6 15 47 0.667 0.444 0.533 |
| L1 | 24 9 3 44 0.727 0.889 0.800 |
| L2 | 26 3 1 50 0.897 0.963 0.929 |
| misses and false alarms of L2 by kind: | |
| case n | 3 wrong=0 |
| clean n | 44 wrong=0 |
| hyphen n | 3 wrong=0 |
| leet n | 3 wrong=0 |
| negated n | 6 wrong=0 |
| plain n | 12 wrong=0 |
| proper_noun n | 3 wrong=3 |
| spaced n | 3 wrong=1 |
| zerowidth n | 3 wrong=0 |
| structural layer on 80 drafts (field checks, exact by construction): | |
| quote a price 41, missing the required tax sentence 10 | |
| restricted requests 9, drafts that fail to refuse 3 | |
| structural checks: precision 1.000, recall 1.000 when the fields are filled | |
| mean draft length 57.6 chars |
Show the full program
# Chapter 4. p40: Three guardrail layers on 80 fictional reply drafts; lexical lint vs structural lint.
# Layer 1 bans five claims. Three matchers of rising strength are scored against ground truth:
# L0 exact substring; L1 casefold + strip zero-width + leet map + hyphen/space collapse;
# L2 = L1 + re-join letters spaced apart + a negation window ("never", "cannot", "no").
# Layer 2 (required sentence) and layer 3 (refusal conditions) are structural checks on fields.
# All drafts, brands and prices are fictional. Ground truth is fixed by construction.
import math
import random
import re
from fractions import Fraction
BANNED = ["guaranteed income", "risk-free", "cures", "miracle results", "act now or lose"]
LEET = str.maketrans({"1": "i", "0": "o", "3": "e", "4": "a", "$": "s"})
ZW = dict.fromkeys(map(ord, ""))
NEG = ("never", "cannot", "can't", "no ", "not ")
def l0(text):
return any(b in text for b in BANNED)
def norm(text):
t = text.translate(ZW).casefold().translate(LEET)
t = t.replace("-", " ")
return re.sub(r"\s+", " ", t)
def l1(text):
t = norm(text)
return any(norm(b) in t for b in BANNED)
def despace(t):
return re.sub(r"\b(?:[a-z] ){2,}[a-z]\b", lambda m: m.group(0).replace(" ", ""), t)
def l2(text):
t = despace(norm(text))
for b in BANNED:
nb = norm(b)
for m in re.finditer(re.escape(nb), t):
window = t[max(0, m.start() - 24):m.start()]
if not any(w in window for w in NEG):
return True
return False
def spaced(p):
return " ".join(p)
def build(rng):
clean = ["Thanks for asking about the Aster deck, it ships in 3 working days.",
"Your booking for the Linden reading room is confirmed for Friday.",
"The studio course has 6 modules and lifetime access to the replays.",
"We can hold the quoted price until the date written on your quote.",
"Here is the size chart for the Juniper card box.",
"The workshop recording will be in your inbox within 48 hours."]
cases = []
for i in range(44):
cases.append(("clean", clean[i % len(clean)], False))
for i in range(12):
b = BANNED[i % 5]
cases.append(("plain", "This plan gives " + b + " for every client.", True))
tricks = [("case", lambda b: b.upper()), ("spaced", lambda b: spaced(b.split()[0]) + " " +
" ".join(b.split()[1:])), ("zerowidth", lambda b: b[:2] + "" + b[2:]),
("leet", lambda b: b.replace("i", "1").replace("e", "3")),
("hyphen", lambda b: b.replace("-", " ").replace(" ", " "))]
for name, f in tricks:
for k in range(3):
b = BANNED[(k * 2 + len(name)) % 5]
cases.append((name, "Join today: " + f(b) + ", trusted by our members.", True))
for k in range(6):
b = BANNED[k % 5]
cases.append(("negated", "We never promise " + b + "; results depend on your own work.", False))
for k in range(3):
cases.append(("proper_noun", "Pick up from 8 Miracle Results Lane, unit {}.".format(k + 2), False))
rng.shuffle(cases)
return cases
def score(cases, f):
tp = sum(1 for _, t, y in cases if y and f(t))
fn = sum(1 for _, t, y in cases if y and not f(t))
fp = sum(1 for _, t, y in cases if not y and f(t))
tn = sum(1 for _, t, y in cases if not y and not f(t))
return tp, fp, fn, tn
def fmt(fr):
return "{:.3f}".format(float(fr))
def structural(rng):
n, priced, disclosed, restricted, refused = 80, 0, 0, 0, 0
miss_disc, miss_ref = 0, 0
for i in range(n):
has_price = rng.random() < 0.45
has_disc = has_price and rng.random() < 0.8
restricted_req = rng.random() < 0.15
did_refuse = restricted_req and rng.random() < 0.7
priced += has_price
disclosed += has_disc
restricted += restricted_req
refused += did_refuse
miss_disc += has_price and not has_disc
miss_ref += restricted_req and not did_refuse
return n, priced, miss_disc, restricted, miss_ref
if __name__ == "__main__":
rng = random.Random(40)
cases = build(rng)
pos = sum(1 for c in cases if c[2])
print("drafts {}, true violations {}, clean or benign {}".format(len(cases), pos, len(cases) - pos))
print("matcher TP FP FN TN precision recall F1")
for name, f in (("L0", l0), ("L1", l1), ("L2", l2)):
tp, fp, fn, tn = score(cases, f)
p = Fraction(tp, tp + fp) if tp + fp else Fraction(0)
r = Fraction(tp, tp + fn)
f1 = 2 * p * r / (p + r) if p + r else Fraction(0)
print("{:7s} {:3d} {:3d} {:3d} {:3d} {:>9s} {:>6s} {:>5s}".format(
name, tp, fp, fn, tn, fmt(p), fmt(r), fmt(f1)))
print("misses and false alarms of L2 by kind:")
kinds = sorted({c[0] for c in cases})
for k in kinds:
sub = [c for c in cases if c[0] == k]
wrong = sum(1 for _, t, y in sub if l2(t) != y)
print(" {:12s} n={:2d} wrong={}".format(k, len(sub), wrong))
n, priced, miss_disc, restricted, miss_ref = structural(random.Random(41))
print("structural layer on {} drafts (field checks, exact by construction):".format(n))
print(" quote a price {}, missing the required tax sentence {}".format(priced, miss_disc))
print(" restricted requests {}, drafts that fail to refuse {}".format(restricted, miss_ref))
print(" structural checks: precision 1.000, recall 1.000 when the fields are filled")
lens = [len(t) for _, t, _ in cases]
print("mean draft length {:.1f} chars".format(math.fsum(lens) / len(lens)))Single source and citation
Fan, Y. (2026). Lossless Distillation of Monographs into Agent Skill Packs, Executable Rule Compilation and Self-Evolving Systems (Version 0.1.0), Chapter 4: Guardrails and Validators: Banned Phrases, Required Sentences, Refusal Conditions, Lint and Golden Samples. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-150 · v0.1.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026150,
author = {Fan, Yafu},
title = {Lossless Distillation of Monographs into Agent Skill Packs, Executable Rule Compilation and Self-Evolving Systems},
year = {2026},
publisher = {5D Library},
edition = {0.1.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Method: Skill Packs, Four Retention Indicators, Evidence Grades and the Four Steps of Distillation
- Chapter 2 · Extraction: Picking the Executable Sentences out of a Manuscript One by One
- Chapter 3 · Compilation: Turning Rule Sentences into Predicates, Decision Tables and Decision Trees
- Chapter 5 · One Specification for the Four Kits: Pricing Calculator, Front-End Style Library, Persuasive-Copy Checker and Customer-Service Closing Pack
- Chapter 6 · Self-Evolution: Feedback Loops, Regression Gates, Drift and the Accumulation of a Skill Library
- Chapter 7 · Governance: Least Privilege, Machine-Checked Red Lines, Disclaimers, Provenance and Rule Retirement
- Chapter 8 · Practice: Distilling a Twelve-Rule Pricing Book into a Pricing Calculator