What does Distilling Books into Skill Packs conclude, in numbers, about “Guardrails and Validators: Banned Phrases, Required Sentences, Refusal Conditions, Lint and Golden Samples”?

Answer: Guardrails come in three sentence types, banned phrases, required sentences and refusal conditions, each paired with replacement wording, trigger predicates and fixed replies; outside text is always data. Lint has a word half and a structural half. On 80 drafts the best word matcher reaches precision 0.897 and recall 0.963, while the structural check is exact when fields are complete, so any rule that can be expressed as fields is not checked with a word list. The unit costs of false alarms and misses set the threshold, which equals \(C_{FP}/(C_{FP}+C_{FN})\) and must be recomputed whenever the base rate moves; otherwise every thousand drafts pay the gap between 6231.4 and 4404.2. Golden samples belong on the boundaries: 18 boundary samples reach a mutation score of 1.000 where 200 random samples reach only 0.767. A clean run only gives a bound, and only 299 clean samples justify saying the error rate is below 0.01. The next chapter builds these checks into the unified specification of the four kits.

Scope of the numbers: All samples are invented; pricing and closing rules are written in general terms and are not legal or compliance advice.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.897, 0.963, 1.000, 80

drafts 80, true violations 27, clean or benign 53
matcher TP FP FN TN precision recall F1
L012 6 15 47 0.667 0.444 0.533
L124 9 3 44 0.727 0.889 0.800
L226 3 1 50 0.897 0.963 0.929
misses and false alarms of L2 by kind:
case n3 wrong=0
clean n44 wrong=0
hyphen n3 wrong=0
leet n3 wrong=0
negated n6 wrong=0
plain n12 wrong=0
proper_noun n3 wrong=3
spaced n3 wrong=1
zerowidth n3 wrong=0
structural layer on 80 drafts (field checks, exact by construction):
quote a price 41, missing the required tax sentence 10
restricted requests 9, drafts that fail to refuse 3
structural checks: precision 1.000, recall 1.000 when the fields are filled
mean draft length 57.6 chars
Show the full program
# Chapter 4. p40: Three guardrail layers on 80 fictional reply drafts; lexical lint vs structural lint.
# Layer 1 bans five claims. Three matchers of rising strength are scored against ground truth:
#   L0 exact substring; L1 casefold + strip zero-width + leet map + hyphen/space collapse;
#   L2 = L1 + re-join letters spaced apart + a negation window ("never", "cannot", "no").
# Layer 2 (required sentence) and layer 3 (refusal conditions) are structural checks on fields.
# All drafts, brands and prices are fictional. Ground truth is fixed by construction.
import math
import random
import re
from fractions import Fraction

BANNED = ["guaranteed income", "risk-free", "cures", "miracle results", "act now or lose"]
LEET = str.maketrans({"1": "i", "0": "o", "3": "e", "4": "a", "$": "s"})
ZW = dict.fromkeys(map(ord, "​‌‍⁠"))
NEG = ("never", "cannot", "can't", "no ", "not ")


def l0(text):
    return any(b in text for b in BANNED)


def norm(text):
    t = text.translate(ZW).casefold().translate(LEET)
    t = t.replace("-", " ")
    return re.sub(r"\s+", " ", t)


def l1(text):
    t = norm(text)
    return any(norm(b) in t for b in BANNED)


def despace(t):
    return re.sub(r"\b(?:[a-z] ){2,}[a-z]\b", lambda m: m.group(0).replace(" ", ""), t)


def l2(text):
    t = despace(norm(text))
    for b in BANNED:
        nb = norm(b)
        for m in re.finditer(re.escape(nb), t):
            window = t[max(0, m.start() - 24):m.start()]
            if not any(w in window for w in NEG):
                return True
    return False


def spaced(p):
    return " ".join(p)


def build(rng):
    clean = ["Thanks for asking about the Aster deck, it ships in 3 working days.",
             "Your booking for the Linden reading room is confirmed for Friday.",
             "The studio course has 6 modules and lifetime access to the replays.",
             "We can hold the quoted price until the date written on your quote.",
             "Here is the size chart for the Juniper card box.",
             "The workshop recording will be in your inbox within 48 hours."]
    cases = []
    for i in range(44):
        cases.append(("clean", clean[i % len(clean)], False))
    for i in range(12):
        b = BANNED[i % 5]
        cases.append(("plain", "This plan gives " + b + " for every client.", True))
    tricks = [("case", lambda b: b.upper()), ("spaced", lambda b: spaced(b.split()[0]) + " " +
              " ".join(b.split()[1:])), ("zerowidth", lambda b: b[:2] + "​" + b[2:]),
              ("leet", lambda b: b.replace("i", "1").replace("e", "3")),
              ("hyphen", lambda b: b.replace("-", " ").replace(" ", "  "))]
    for name, f in tricks:
        for k in range(3):
            b = BANNED[(k * 2 + len(name)) % 5]
            cases.append((name, "Join today: " + f(b) + ", trusted by our members.", True))
    for k in range(6):
        b = BANNED[k % 5]
        cases.append(("negated", "We never promise " + b + "; results depend on your own work.", False))
    for k in range(3):
        cases.append(("proper_noun", "Pick up from 8 Miracle Results Lane, unit {}.".format(k + 2), False))
    rng.shuffle(cases)
    return cases


def score(cases, f):
    tp = sum(1 for _, t, y in cases if y and f(t))
    fn = sum(1 for _, t, y in cases if y and not f(t))
    fp = sum(1 for _, t, y in cases if not y and f(t))
    tn = sum(1 for _, t, y in cases if not y and not f(t))
    return tp, fp, fn, tn


def fmt(fr):
    return "{:.3f}".format(float(fr))


def structural(rng):
    n, priced, disclosed, restricted, refused = 80, 0, 0, 0, 0
    miss_disc, miss_ref = 0, 0
    for i in range(n):
        has_price = rng.random() < 0.45
        has_disc = has_price and rng.random() < 0.8
        restricted_req = rng.random() < 0.15
        did_refuse = restricted_req and rng.random() < 0.7
        priced += has_price
        disclosed += has_disc
        restricted += restricted_req
        refused += did_refuse
        miss_disc += has_price and not has_disc
        miss_ref += restricted_req and not did_refuse
    return n, priced, miss_disc, restricted, miss_ref


if __name__ == "__main__":
    rng = random.Random(40)
    cases = build(rng)
    pos = sum(1 for c in cases if c[2])
    print("drafts {}, true violations {}, clean or benign {}".format(len(cases), pos, len(cases) - pos))
    print("matcher  TP  FP  FN  TN  precision  recall  F1")
    for name, f in (("L0", l0), ("L1", l1), ("L2", l2)):
        tp, fp, fn, tn = score(cases, f)
        p = Fraction(tp, tp + fp) if tp + fp else Fraction(0)
        r = Fraction(tp, tp + fn)
        f1 = 2 * p * r / (p + r) if p + r else Fraction(0)
        print("{:7s} {:3d} {:3d} {:3d} {:3d}  {:>9s}  {:>6s}  {:>5s}".format(
            name, tp, fp, fn, tn, fmt(p), fmt(r), fmt(f1)))
    print("misses and false alarms of L2 by kind:")
    kinds = sorted({c[0] for c in cases})
    for k in kinds:
        sub = [c for c in cases if c[0] == k]
        wrong = sum(1 for _, t, y in sub if l2(t) != y)
        print("   {:12s} n={:2d} wrong={}".format(k, len(sub), wrong))
    n, priced, miss_disc, restricted, miss_ref = structural(random.Random(41))
    print("structural layer on {} drafts (field checks, exact by construction):".format(n))
    print("   quote a price {}, missing the required tax sentence {}".format(priced, miss_disc))
    print("   restricted requests {}, drafts that fail to refuse {}".format(restricted, miss_ref))
    print("   structural checks: precision 1.000, recall 1.000 when the fields are filled")
    lens = [len(t) for _, t, _ in cases]
    print("mean draft length {:.1f} chars".format(math.fsum(lens) / len(lens)))

Single source and citation

Fan, Y. (2026). Lossless Distillation of Monographs into Agent Skill Packs, Executable Rule Compilation and Self-Evolving Systems (Version 0.1.0), Chapter 4: Guardrails and Validators: Banned Phrases, Required Sentences, Refusal Conditions, Lint and Golden Samples. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-150 · v0.1.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026150,
  author    = {Fan, Yafu},
  title     = {Lossless Distillation of Monographs into Agent Skill Packs, Executable Rule Compilation and Self-Evolving Systems},
  year      = {2026},
  publisher = {5D Library},
  edition   = {0.1.0},
  note      = {DOI pending}
}

Other chapter answers in this book