关于「抽取:把书稿里能执行的句子一句一句拣出来」,《专著蒸馏与技能包工程》给出的定量结论是什么?
结论:规则句有八种句型:条件—动作、阈值、禁忌、次序、例外、公式、拒绝、定义,标注框是第九种信号。在 30 句的虚构报价手册上,识框的提示词抽取器中文精确率 0.8966、召回率 0.8387,英文精确率 0.8333、召回率 0.9677;标注框信号不花召回率就拿掉了误报,中英并集只漏掉一句不带任何提示词的隐性条件。中文数量词要先换算,换算不了的要拒绝。召回率的结论受金标准条数限制:目标 0.90 而一条不漏,至少要 35 条金标准。两位标注者的 kappa 受规则句比例左右,比较不同书稿时要先看比例。抽取的产物是一张逐栏填满的抽取单,第三章从这张单子出发,把规则句编译成可以执行的谓词。
数据口径:书中样本全部虚构,定价与成交规则只写一般框架,不是法律或合规意见。 全书按证据分级标注每一条说法,以《荀子·劝学》考辨工具与积累,所有计算由书中程序生成,作者的判断一律写进假说框并写明什么结果会推翻它。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.8333, 0.8387, 0.8966, 0.90, 0.9677, 30
| recall with 95% Wilson interval | |
| zh overall | 26/31 = 0.8387 [0.6737, 0.9291] |
| en overall | 30/31 = 0.9677 [0.8381, 0.9943] |
| zh COND | 3/ 6 = 0.5000 [0.1876, 0.8124] |
| zh THRESH | 6/ 7 = 0.8571 [0.4869, 0.9743] |
| PROHIB | 3/ 3 = 1.0000 [0.4385, 1.0000] |
| EXCEPT | 2/ 2 = 1.0000 [0.3424, 1.0000] |
| zh recall: micro 0.8387, macro over 8 types 0.8780 | |
| en recall: micro 0.9677, macro over 8 types 0.9792 | |
| zh: P 0.8966 R 0.8387 F1 0.8667 F2 0.8497 F0.5 0.8844 | |
| en: P 0.8333 R 0.9677 F1 0.8955 F2 0.9375 F0.5 0.8571 | |
| gold rules needed so that the Wilson lower bound on recall reaches the target | |
| target 0.80 | 0 miss -> n 16; 1 miss -> n 25; 2 miss -> n 33; 3 miss -> n 40 |
| target 0.90 | 0 miss -> n 35; 1 miss -> n 53; 2 miss -> n 69; 3 miss -> n 84 |
| target 0.95 | 0 miss -> n 73; 1 miss -> n 110; 2 miss -> n 142; 3 miss -> n 173 |
| zero misses in n gold rules: one-sided 95% upper bound on miss rate | 1 - 0.05^(1/n) |
| n | 10: exact 0.2589, rule of three 3/n 0.3000 |
| n | 30: exact 0.0950, rule of three 3/n 0.1000 |
| n 100: exact 0.0295, rule of three 3/n 0.0300 | |
| n 300: exact 0.0099, rule of three 3/n 0.0100 |
查看完整验算程序
# Chapter 2. p21: Wilson intervals for extraction recall, micro vs macro averages, and audit sizes.
# Counts are the box-aware results printed by p20 (TP, FN per type; TP, FP overall).
# Wilson score interval: centre (k + z^2/2) / (n + z^2), half-width z sqrt(k(n-k)/n + z^2/4) / (n + z^2).
# F-beta = (1 + b^2) P R / (b^2 P + R); b = 2 weighs a missed rule four times a false one.
import math
from fractions import Fraction
Z = 1.959964 # two-sided 95 %
def wilson(k, n, z=Z):
c = (k + z * z / 2) / (n + z * z)
h = z * math.sqrt(k * (n - k) / n + z * z / 4) / (n + z * z)
return c - h, c + h
def fbeta(p, r, b):
return (1 + b * b) * p * r / (b * b * p + r)
# per type (TP, FN) from p20, box-aware
ZH = {"COND": (3, 3), "THRESH": (6, 1), "PROHIB": (3, 0), "ORDER": (4, 0), "EXCEPT": (2, 0),
"FORMULA": (3, 0), "REFUSE": (3, 0), "DEF": (2, 1)}
EN = {"COND": (5, 1), "THRESH": (7, 0), "PROHIB": (3, 0), "ORDER": (4, 0), "EXCEPT": (2, 0),
"FORMULA": (3, 0), "REFUSE": (3, 0), "DEF": (3, 0)}
OVERALL = {"zh": (26, 3, 5), "en": (30, 6, 1)} # TP, FP, FN
if __name__ == "__main__":
print(" recall with 95% Wilson interval")
for name, (k, n) in [("zh overall", (26, 31)), ("en overall", (30, 31)), ("zh COND", (3, 6)),
("zh THRESH", (6, 7)), ("PROHIB", (3, 3)), ("EXCEPT", (2, 2))]:
lo, hi = wilson(k, n)
print(" {:11s} {:2d}/{:2d} = {:.4f} [{:.4f}, {:.4f}]".format(name, k, n, k / n, lo, hi))
for lang, table in (("zh", ZH), ("en", EN)):
tp = sum(a for a, _ in table.values())
fn = sum(b for _, b in table.values())
micro = Fraction(tp, tp + fn)
macro = sum(Fraction(a, a + b) for a, b in table.values()) / len(table)
print(" {} recall: micro {:.4f}, macro over 8 types {:.4f}".format(
lang, float(micro), float(macro)))
for lang, (tp, fp, fn) in OVERALL.items():
p, r = tp / (tp + fp), tp / (tp + fn)
print(" {}: P {:.4f} R {:.4f} F1 {:.4f} F2 {:.4f} F0.5 {:.4f}".format(
lang, p, r, fbeta(p, r, 1), fbeta(p, r, 2), fbeta(p, r, 0.5)))
print(" gold rules needed so that the Wilson lower bound on recall reaches the target")
for target in (0.80, 0.90, 0.95):
row = []
for misses in (0, 1, 2, 3):
n = misses + 1
while wilson(n - misses, n)[0] < target:
n += 1
row.append("{} miss -> n {}".format(misses, n))
print(" target {:.2f}: {}".format(target, "; ".join(row)))
print(" zero misses in n gold rules: one-sided 95% upper bound on miss rate = 1 - 0.05^(1/n)")
for n in (10, 30, 100, 300):
print(" n {:3d}: exact {:.4f}, rule of three 3/n {:.4f}".format(n, 1 - 0.05 ** (1 / n), 3 / n))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《专著向智能体技能包无损蒸馏、规则可执行化编译与系统自进化工程》第 2 章「抽取:把书稿里能执行的句子一句一句拣出来」(版本 0.1.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-150 · v0.1.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026150,
author = {Fan, Yafu},
title = {Lossless Distillation of Monographs into Agent Skill Packs, Executable Rule Compilation and Self-Evolving Systems},
year = {2026},
publisher = {5D Library},
edition = {0.1.0},
note = {DOI pending}
}