Three Lights for Handing Over between Bot and Human in Private Customer Service: Misclassification Loss Matrices, Chow's Reject Rule and Optimal Cut Points on Calibrated Confidence: chapter answers
Once a bot joins a shop's private customer-service channel, the usual practice is to set a confidence threshold by accuracy and let the bot answer whenever it is high enough. The commissioning claim says that a threshold set by accuracy alone is wrong, because with a reject option the optimal threshold is decided by the ratio of misclassification costs (Chow 1970). Writing the claim as computable models in standard-library programs, the book corrects the blueprint in three places. Optimal recognition with a rejection channel already appears in Chow's 1957 paper, and the 1970 paper gives the g…
DOI pending · monograph page goes live after publication · 5DL-B-266 · 中文版
- Chapter 1 · The Blind Spot of Accuracy: Chow's Reject Rule and the Cost Ratio
Now read down. When a wrong answer costs 8 times as much as staff time, Chow's threshold moves to 0.8750 and the loss is 389.2 yuan; A1 stays put and loses 558.7 yuan; A2, with its threshold of 0.8170, is not far from t… - Chapter 2 · A Six-Cell Loss Matrix: Bayes Risk in Three Zones and Two Cut Points
A guess at 0.50 of the true value, 25.0 yuan, loses an extra 17.5 yuan per hundred conversations, 7.3 % of the optimal loss; a guess at 2.00 times, 100.0 yuan, loses an extra 23.0 yuan, or 9.6 %. A guess off by about a… - Chapter 3 · Cost Curves: Why the More Accurate Bot Can Cost More
Chapter 3 compares two bots on cost curves; the less accurate bot that rejects better is cheaper above a cost ratio of 4.14. - Chapter 4 · Confidence Is Not Probability: Isotonic Calibration and the Price of Few Labels
The optimal loss under the true probability is 240.0 yuan per hundred conversations. With only 50 labels, isotonic regression loses an extra 38.4 yuan on average and Platt scaling 12.9 yuan; with 100 labels, 22.5 and 6.… - Chapter 5 · Big Sales and Drift: Bayesian Correction for Prior Shift and the Sampling Error of Cut Points
Chapter 5 treats the prior shift of a big sale: when the true share of right answers falls to 0.600, the bot's mean self-reported chance is still 0.843. - Chapter 6 · Red Lights Beyond Price: Stop Means Stop, Vulnerable Customers Go to Staff, and the Staffing Cap
Without a cap the optimal cut points are those of Chapter 2, 0.400 and 0.950, with a load of 22.1 units and a loss of 295.2 yuan per hundred conversations, which here includes the staff cost of the hard-rule conversatio… - Chapter 7 · Setting the Doubtful Aside, and Bayes: Two Classical Texts on Being Unsure
Chapter 7 examines the Analects' "hear much, set aside the doubtful, speak with care about the rest, and you will seldom err" and Bayes's paper of 1763: with 19 right out of 20 reviewed, the posterior mean is only 0.909… - Chapter 8 · The Monthly Recalculation Loop, the Distilled Tool and a Falsification Plan
Chapter 8 gives a monthly recalculation routine with an event rule for version changes, the distilled "Three-Light Cut-Point Calculator", a pre-registered test needing 1177 conversations per group, and the conditions th…
Illustrative model calculation, not a live quote