关于「置信度不是概率:等渗回归校准与少样本的代价」,《三级灯号切分点》给出的定量结论是什么?

结论:真实概率下的最优损失是每百段 240.0 元。只有 50 段标注时,等渗回归平均多损失 38.4 元,普拉特缩放多损失 12.9 元;100 段时分别是 22.5 元与 6.6 元;300 段时 7.6 元与 1.9 元;1,000 段时 3.6 元与 0.8 元;3,000 段时 1.8 元与 0.3 元。校准值与真实概率的平均差同样如此:50 段时等渗回归 0.0687、普拉特 0.0382,3,000 段时降到 0.0145 与 0.0051。

数据口径:本书用纯标准库程序把命题写成可计算的模型,并修正蓝图三处:带拒识通道的最优识别早见于 Chow 1957 年的论文,1970 年的论文给出误差与拒判的一般关系;规则只对校准过的概率成立,把成本比算出的切分点直接套在过度自信机器人的自报分数上,每百段对话损失 265.4 元,等渗回归校准后 239.8 元;私域客服有「机器人起草、真人过目」的第三种动作,需要六格损失矩阵与两条切分点,示意参数下答对概率低于 0.4000 亮红灯、不低于 0.9500 亮绿灯。 书中参数为示意,对话全部虚构;全部计算由书中程序生成;程序不读写任何文件,不连接任何客服系统;本书是非个人化经营建议,任何线上灯号的改动须由主事者人工复核放行。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0051, 0.0145, 0.0382, 0.0687, 0.3, 0.8

oracle cost per 100 chats: 240.0
niso regretPlatt regretiso |q-p|Platt |q-p|
5038.412.90.06870.0382
10022.56.60.05600.0297
3007.61.90.03360.0151
1,0003.60.80.02180.0097
3,0001.80.30.01450.0051
查看完整验算程序
# Chapter 4. p43: with few labels, isotonic overfits; Platt holds up.
# Model (illustrative): the p41 world and the p21 loss matrix. For each
# training size n, 40 repeats (seed 20261113) fit isotonic and Platt
# maps; each is scored on one fixed set of 4000 chats by the expected
# loss under the true p (yuan per 100 chats, regret over the oracle)
# and by the mean absolute gap |q - p| between map and truth. Platt
# uses smoothed targets (n+ + 1) / (n+ + 2) and 1 / (n- + 2). The
# Platt family here contains the truth (b = 1 / 1.6), which favours it.
import bisect
import math
import random

rng = random.Random(20261113)
LW, CA, M, CR = 50.0, 2.0, 0.8, 8.0
P_HI = 1 - CA / (M * LW)
P_LO = 1 - (CR - CA) / ((1 - M) * LW)


def draw(n):
    out = []
    for _ in range(n):
        z = rng.gauss(3.0, 1.6)
        p = 1 / (1 + math.exp(-z))
        y = 1 if rng.random() < p else 0
        out.append((1 / (1 + math.exp(-1.6 * z)), y, p))
    return out


def pav(data):
    blocks = []
    for s, y in sorted(data):
        blocks.append([s, float(y), 1])
        while (len(blocks) > 1 and blocks[-2][1] / blocks[-2][2]
               >= blocks[-1][1] / blocks[-1][2]):
            s2, t2, n2 = blocks.pop()
            blocks[-1][1] = math.fsum([blocks[-1][1], t2])
            blocks[-1][2] += n2
    starts, vals = [b[0] for b in blocks], [b[1] / b[2] for b in blocks]
    return lambda s: vals[max(0, bisect.bisect_right(starts, s) - 1)]


def logit(s):
    s = min(max(s, 1e-9), 1 - 1e-9)
    return math.log(s / (1 - s))


def sig(u):
    if u >= 0:
        return 1 / (1 + math.exp(-u))
    e = math.exp(u)
    return e / (1 + e)


def platt(data):
    # smoothed targets as in Platt's recipe keep the fit finite
    npos = len([1 for _, y in data if y == 1])
    tp, tn = (npos + 1) / (npos + 2), 1 / (len(data) - npos + 2)
    a, b = 0.0, 1.0
    xs = [(logit(s), tp if y == 1 else tn) for s, y in data]

    def nll(a, b):
        return -math.fsum(t * math.log(sig(a + b * x) + 1e-300)
                          + (1 - t) * math.log(sig(-a - b * x) + 1e-300)
                          for x, t in xs)

    for _ in range(25):
        g0, g1, h00, h01, h11 = [], [], [], [], []
        for x, y in xs:
            q = sig(a + b * x)
            v = q * (1 - q) + 1e-12
            g0.append(y - q)
            g1.append((y - q) * x)
            h00.append(v)
            h01.append(v * x)
            h11.append(v * x * x)
        G0, G1 = math.fsum(g0), math.fsum(g1)
        H00, H01, H11 = math.fsum(h00), math.fsum(h01), math.fsum(h11)
        det = H00 * H11 - H01 * H01
        da = (H11 * G0 - H01 * G1) / det
        db = (H00 * G1 - H01 * G0) / det
        step, f0 = 1.0, nll(a, b)
        while step > 1e-6 and nll(a + step * da, b + step * db) > f0:
            step /= 2  # damped Newton: halve until the fit improves
        a, b = a + step * da, b + step * db
    return lambda s: sig(a + b * logit(s))


def risk(q, p):
    if q >= P_HI:
        return (1 - p) * LW
    if q >= P_LO:
        return CA + (1 - p) * (1 - M) * LW
    return CR


test = draw(4000)
oracle = 100 * math.fsum(risk(p, p) for _, _, p in test) / len(test)
print("oracle cost per 100 chats: %.1f" % oracle)
print("     n  iso regret  Platt regret  iso |q-p|  Platt |q-p|")
for n in (50, 100, 300, 1000, 3000):
    ri, rp, gi, gp = [], [], [], []
    for _ in range(40):
        tr = [(s, y) for s, y, _ in draw(n)]
        fi, fp = pav(tr), platt(tr)
        qi = [fi(s) for s, _, _ in test]
        qp = [fp(s) for s, _, _ in test]
        ri.append(100 * math.fsum(risk(q, r[2]) for q, r in zip(qi, test))
                  / len(test) - oracle)
        rp.append(100 * math.fsum(risk(q, r[2]) for q, r in zip(qp, test))
                  / len(test) - oracle)
        gi.append(math.fsum(abs(q - r[2]) for q, r in zip(qi, test))
                  / len(test))
        gp.append(math.fsum(abs(q - r[2]) for q, r in zip(qp, test))
                  / len(test))
    print("%6s  %10.1f  %12.1f  %9.4f  %11.4f" % (
        format(n, ","), math.fsum(ri) / 40, math.fsum(rp) / 40,
        math.fsum(gi) / 40, math.fsum(gp) / 40))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《私域客服人机交接三级灯号精算、误判损失矩阵与最优置信度切分点母典》第 4 章「置信度不是概率:等渗回归校准与少样本的代价」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-266 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026266,
  author    = {Fan, Yafu},
  title     = {Three Lights for Handing Over between Bot and Human in Private Customer Service: Misclassification Loss Matrices, Chow's Reject Rule and Optimal Cut Points on Calibrated Confidence},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论