关于「置信度不是概率:等渗回归校准与少样本的代价」,《三级灯号切分点》给出的定量结论是什么?
结论:真实概率下的最优损失是每百段 240.0 元。只有 50 段标注时,等渗回归平均多损失 38.4 元,普拉特缩放多损失 12.9 元;100 段时分别是 22.5 元与 6.6 元;300 段时 7.6 元与 1.9 元;1,000 段时 3.6 元与 0.8 元;3,000 段时 1.8 元与 0.3 元。校准值与真实概率的平均差同样如此:50 段时等渗回归 0.0687、普拉特 0.0382,3,000 段时降到 0.0145 与 0.0051。
数据口径:本书用纯标准库程序把命题写成可计算的模型,并修正蓝图三处:带拒识通道的最优识别早见于 Chow 1957 年的论文,1970 年的论文给出误差与拒判的一般关系;规则只对校准过的概率成立,把成本比算出的切分点直接套在过度自信机器人的自报分数上,每百段对话损失 265.4 元,等渗回归校准后 239.8 元;私域客服有「机器人起草、真人过目」的第三种动作,需要六格损失矩阵与两条切分点,示意参数下答对概率低于 0.4000 亮红灯、不低于 0.9500 亮绿灯。 书中参数为示意,对话全部虚构;全部计算由书中程序生成;程序不读写任何文件,不连接任何客服系统;本书是非个人化经营建议,任何线上灯号的改动须由主事者人工复核放行。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0051, 0.0145, 0.0382, 0.0687, 0.3, 0.8
| oracle cost per 100 chats: 240.0 | ||||
| n | iso regret | Platt regret | iso |q-p| | Platt |q-p| |
| 50 | 38.4 | 12.9 | 0.0687 | 0.0382 |
| 100 | 22.5 | 6.6 | 0.0560 | 0.0297 |
| 300 | 7.6 | 1.9 | 0.0336 | 0.0151 |
| 1,000 | 3.6 | 0.8 | 0.0218 | 0.0097 |
| 3,000 | 1.8 | 0.3 | 0.0145 | 0.0051 |
查看完整验算程序
# Chapter 4. p43: with few labels, isotonic overfits; Platt holds up.
# Model (illustrative): the p41 world and the p21 loss matrix. For each
# training size n, 40 repeats (seed 20261113) fit isotonic and Platt
# maps; each is scored on one fixed set of 4000 chats by the expected
# loss under the true p (yuan per 100 chats, regret over the oracle)
# and by the mean absolute gap |q - p| between map and truth. Platt
# uses smoothed targets (n+ + 1) / (n+ + 2) and 1 / (n- + 2). The
# Platt family here contains the truth (b = 1 / 1.6), which favours it.
import bisect
import math
import random
rng = random.Random(20261113)
LW, CA, M, CR = 50.0, 2.0, 0.8, 8.0
P_HI = 1 - CA / (M * LW)
P_LO = 1 - (CR - CA) / ((1 - M) * LW)
def draw(n):
out = []
for _ in range(n):
z = rng.gauss(3.0, 1.6)
p = 1 / (1 + math.exp(-z))
y = 1 if rng.random() < p else 0
out.append((1 / (1 + math.exp(-1.6 * z)), y, p))
return out
def pav(data):
blocks = []
for s, y in sorted(data):
blocks.append([s, float(y), 1])
while (len(blocks) > 1 and blocks[-2][1] / blocks[-2][2]
>= blocks[-1][1] / blocks[-1][2]):
s2, t2, n2 = blocks.pop()
blocks[-1][1] = math.fsum([blocks[-1][1], t2])
blocks[-1][2] += n2
starts, vals = [b[0] for b in blocks], [b[1] / b[2] for b in blocks]
return lambda s: vals[max(0, bisect.bisect_right(starts, s) - 1)]
def logit(s):
s = min(max(s, 1e-9), 1 - 1e-9)
return math.log(s / (1 - s))
def sig(u):
if u >= 0:
return 1 / (1 + math.exp(-u))
e = math.exp(u)
return e / (1 + e)
def platt(data):
# smoothed targets as in Platt's recipe keep the fit finite
npos = len([1 for _, y in data if y == 1])
tp, tn = (npos + 1) / (npos + 2), 1 / (len(data) - npos + 2)
a, b = 0.0, 1.0
xs = [(logit(s), tp if y == 1 else tn) for s, y in data]
def nll(a, b):
return -math.fsum(t * math.log(sig(a + b * x) + 1e-300)
+ (1 - t) * math.log(sig(-a - b * x) + 1e-300)
for x, t in xs)
for _ in range(25):
g0, g1, h00, h01, h11 = [], [], [], [], []
for x, y in xs:
q = sig(a + b * x)
v = q * (1 - q) + 1e-12
g0.append(y - q)
g1.append((y - q) * x)
h00.append(v)
h01.append(v * x)
h11.append(v * x * x)
G0, G1 = math.fsum(g0), math.fsum(g1)
H00, H01, H11 = math.fsum(h00), math.fsum(h01), math.fsum(h11)
det = H00 * H11 - H01 * H01
da = (H11 * G0 - H01 * G1) / det
db = (H00 * G1 - H01 * G0) / det
step, f0 = 1.0, nll(a, b)
while step > 1e-6 and nll(a + step * da, b + step * db) > f0:
step /= 2 # damped Newton: halve until the fit improves
a, b = a + step * da, b + step * db
return lambda s: sig(a + b * logit(s))
def risk(q, p):
if q >= P_HI:
return (1 - p) * LW
if q >= P_LO:
return CA + (1 - p) * (1 - M) * LW
return CR
test = draw(4000)
oracle = 100 * math.fsum(risk(p, p) for _, _, p in test) / len(test)
print("oracle cost per 100 chats: %.1f" % oracle)
print(" n iso regret Platt regret iso |q-p| Platt |q-p|")
for n in (50, 100, 300, 1000, 3000):
ri, rp, gi, gp = [], [], [], []
for _ in range(40):
tr = [(s, y) for s, y, _ in draw(n)]
fi, fp = pav(tr), platt(tr)
qi = [fi(s) for s, _, _ in test]
qp = [fp(s) for s, _, _ in test]
ri.append(100 * math.fsum(risk(q, r[2]) for q, r in zip(qi, test))
/ len(test) - oracle)
rp.append(100 * math.fsum(risk(q, r[2]) for q, r in zip(qp, test))
/ len(test) - oracle)
gi.append(math.fsum(abs(q - r[2]) for q, r in zip(qi, test))
/ len(test))
gp.append(math.fsum(abs(q - r[2]) for q, r in zip(qp, test))
/ len(test))
print("%6s %10.1f %12.1f %9.4f %11.4f" % (
format(n, ","), math.fsum(ri) / 40, math.fsum(rp) / 40,
math.fsum(gi) / 40, math.fsum(gp) / 40))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《私域客服人机交接三级灯号精算、误判损失矩阵与最优置信度切分点母典》第 4 章「置信度不是概率:等渗回归校准与少样本的代价」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-266 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026266,
author = {Fan, Yafu},
title = {Three Lights for Handing Over between Bot and Human in Private Customer Service: Misclassification Loss Matrices, Chow's Reject Rule and Optimal Cut Points on Calibrated Confidence},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}