What does Three-Light Cut Points conclude, in numbers, about “The Blind Spot of Accuracy: Chow's Reject Rule and the Cost Ratio”?
Answer: Now read down. When a wrong answer costs 8 times as much as staff time, Chow's threshold moves to 0.8750 and the loss is 389.2 yuan; A1 stays put and loses 558.7 yuan; A2, with its threshold of 0.8170, is not far from the optimum and loses 402.8 yuan. At 16 times, Chow's threshold is 0.9375 with a loss of 514.8 yuan, A1 climbs to 1093.1 yuan and A2 to 666.2 yuan, so the worse of the two accuracy yardsticks pays 578.3 yuan more. At 32 times the best threshold is 0.9688, and A1 loses 2161.8 yuan, more than three times the optimum of 623.4 yuan.
Scope of the numbers: Private customer service has a third action, a bot draft checked by a person, so it needs a six-cell loss matrix with two cut points; under illustrative parameters red lights below a chance of being right of 0.4000 and green from 0.9500. Parameters are illustrative, all conversations are fictional, and all numbers come from the book's programs, which read and write no file and connect to no customer-service system.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.8170, 0.8750, 0.9375, 0.9688, 1093.1, 16
| bot accuracy if it answered everything: 0.897 | |||||
| A1 cut 0.500 (accuracy rule); A2 cut 0.8170 (answered acc 0.95) | |||||
| LW/CR | Chow cut | cost Chow | cost A1 | cost A2 | worst regret |
| 2 | 0.5000 | 158.0 | 158.0 | 205.2 | 47.2 |
| 4 | 0.7500 | 264.4 | 291.6 | 271.0 | 27.1 |
| 8 | 0.8750 | 389.2 | 558.7 | 402.8 | 169.5 |
| 16 | 0.9375 | 514.8 | 1093.1 | 666.2 | 578.3 |
| 32 | 0.9688 | 623.4 | 2161.8 | 1193.2 | 1538.5 |
Show the full program
# Chapter 1. p11: a cut point tuned for accuracy ignores the cost ratio.
# Model (illustrative): a latent case difficulty z ~ N(3.0, 1.6); the
# bot answers correctly with calibrated probability p = 1 / (1 + e^-z).
# Two lights only: green (bot answers) or red (a human takes over at
# CR = 8 yuan). A wrong bot answer costs LW yuan. Chow's rule answers
# when p >= 1 - CR / LW. Two accuracy rules are shown for contrast:
# A1 answers when p >= 0.5 (best accuracy of a right/wrong predictor),
# A2 picks the lowest cut whose answered accuracy reaches 0.95.
# Exact quadrature on 4001 nodes; costs are yuan per 100 chats.
import math
MU, SD, CR = 3.0, 1.6, 8.0
def nodes(n=4001):
zs = [MU + SD * (-8.0 + 16.0 * i / (n - 1)) for i in range(n)]
ws = [math.exp(-0.5 * ((z - MU) / SD) ** 2) for z in zs]
tot = math.fsum(ws)
return [(1.0 / (1.0 + math.exp(-z)), w / tot) for z, w in zip(zs, ws)]
PW = nodes()
def cost(theta, lw):
# expected loss per chat: answer -> (1 - p) LW, hand over -> CR
terms = [w * ((1 - p) * lw if p >= theta else CR) for p, w in PW]
return 100 * math.fsum(terms)
def answered_acc(theta):
num = math.fsum(w * p for p, w in PW if p >= theta)
den = math.fsum(w for p, w in PW if p >= theta)
return num / den
acc = math.fsum(w * p for p, w in PW)
a2 = next(0.5 + k / 2000 for k in range(1000)
if answered_acc(0.5 + k / 2000) >= 0.95)
print("bot accuracy if it answered everything: %.3f" % acc)
print("A1 cut 0.500 (accuracy rule); A2 cut %.4f (answered acc 0.95)"
% a2)
print(" LW/CR Chow cut cost Chow cost A1 cost A2 worst regret")
for ratio in (2, 4, 8, 16, 32):
lw = ratio * CR
t = 1 - CR / lw
c0, c1, c2 = cost(t, lw), cost(0.5, lw), cost(a2, lw)
print("%6d %8.4f %9.1f %7.1f %7.1f %12.1f"
% (ratio, t, c0, c1, c2, max(c1, c2) - c0))Single source and citation
Fan, Y. (2026). Three Lights for Handing Over between Bot and Human in Private Customer Service: Misclassification Loss Matrices, Chow's Reject Rule and Optimal Cut Points on Calibrated Confidence (Version 1.0.0), Chapter 1: The Blind Spot of Accuracy: Chow's Reject Rule and the Cost Ratio. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-266 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026266,
author = {Fan, Yafu},
title = {Three Lights for Handing Over between Bot and Human in Private Customer Service: Misclassification Loss Matrices, Chow's Reject Rule and Optimal Cut Points on Calibrated Confidence},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 2 · A Six-Cell Loss Matrix: Bayes Risk in Three Zones and Two Cut Points
- Chapter 3 · Cost Curves: Why the More Accurate Bot Can Cost More
- Chapter 4 · Confidence Is Not Probability: Isotonic Calibration and the Price of Few Labels
- Chapter 5 · Big Sales and Drift: Bayesian Correction for Prior Shift and the Sampling Error of Cut Points
- Chapter 6 · Red Lights Beyond Price: Stop Means Stop, Vulnerable Customers Go to Staff, and the Staffing Cap
- Chapter 7 · Setting the Doubtful Aside, and Bayes: Two Classical Texts on Being Unsure
- Chapter 8 · The Monthly Recalculation Loop, the Distilled Tool and a Falsification Plan