What does Active Learning for Intent Labels conclude, in numbers, about “Closing the Loop: a Labeling Queue for Customer Intents and a Plan to Refute the Claim”?
Answer: Chapter 8 gives a labeling queue for customer intents, the distilled Labeling Priority Sorter and a pre-registered test; the classifier only routes and suggests a hand-over and takes no adverse decision about a customer automatically. Marked boxes separate published evidence, runnable Python calculations (23 programs), classical sources and the author's hypotheses.
Scope of the numbers: Intent priors, word probabilities and costs are illustrative; all calculations are generated by the book's programs; the book is general business analysis, not individualised business advice. Marked boxes separate published evidence, runnable Python calculations (23 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python only
| ok | hi my name is [NAME], where is order [ORDER] |
|---|---|
| ok | refund please, card [CARD] was charged twice |
| ok | call me on [PHONE] about the wrong size |
| ok | email [EMAIL], parcel [ORDER] is late |
| ok | I want to speak to a manager, phone [PHONE] |
| ok | exchange size M for L please, thanks |
| replacements: EMAIL=1 ORDER=2 CARD=1 PHONE=2 NAME=1 | |
| messages with a digit run of 4 or more left: 0 of 6 |
Show the full program
# Chapter 8. p82: de-identify messages before they enter the label queue.
# 只用店内客服对话;入队前用正则把邮箱、电话、订单号、长数字串与
# 自报姓名替换为占位标记,再检查是否残留 4 位以上连续数字。
# 下列消息为虚构样例(example.com 域名、虚构号码),不含任何真实数据。
import re
RULES = (
("EMAIL", re.compile(r"[\w.+-]+@[\w-]+(\.[\w-]+)+")),
("ORDER", re.compile(r"\b(?:ORD|SO)-?\d{4,}(?:-\d+)*\b", re.I)),
("CARD", re.compile(r"\b\d(?:[ -]?\d){12,18}\b")),
("PHONE", re.compile(r"(?<!\w)\+?\d[\d -]{7,14}\d(?!\w)")),
("NAME", re.compile(r"(?<=\bmy name is )[A-Z][a-z]+", re.I)),
)
MSGS = (
"hi my name is Dana, where is order ORD-20261113-0042",
"refund please, card 4111 1111 1111 1111 was charged twice",
"call me on +60 12-345 6789 about the wrong size",
"email dana.k@example.com, parcel SO998877 is late",
"I want to speak to a manager, phone 0123456789",
"exchange size M for L please, thanks",
)
counts = {name: 0 for name, _ in RULES}
leaks = 0
for m in MSGS:
out = m
for name, rx in RULES:
out, k = rx.subn("[" + name + "]", out)
counts[name] += k
left = re.findall(r"\d{4,}", out)
leaks += 1 if left else 0
print(("ok " if not left else "LEAK") + " " + out)
print("replacements: " + " ".join(
"{}={}".format(name, counts[name]) for name, _ in RULES))
print("messages with a digit run of 4 or more left: {} of {}".format(
leaks, len(MSGS)))Single source and citation
Fan, Y. (2026). Active Learning and Uncertainty Sampling: the Fewest Human Labels a Customer-Intent Classifier Needs to Reach Its Target (Version 1.0.0), Chapter 8: Closing the Loop: a Labeling Queue for Customer Intents and a Plan to Refute the Claim. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-312 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026312,
author = {Fan, Yafu},
title = {Active Learning and Uncertainty Sampling: the Fewest Human Labels a Customer-Intent Classifier Needs to Reach Its Target},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Thousands versus Hundreds: the Counterintuitive Ledger of Uncertainty Sampling
- Chapter 2 · A Hand-Written Multinomial Logistic Regression: Stochastic Gradient Descent on Integer Bag-of-Words Features
- Chapter 3 · Three Rulers of Doubt: Least Confidence, Smallest Margin and Entropy
- Chapter 4 · Sampling Bias: Picked Messages No Longer Stand for the Traffic
- Chapter 5 · Importance Weighting and Prior Adjustment: Computing the Bias Away
- Chapter 6 · Labeling Budgets, Stopping Rules and Audit Sample Sizes
- Chapter 7 · Eastern and Western Classics: the Xueji's Easy-First Order and Bacon's Crucial Instances