What does Three-Light Cut Points conclude, in numbers, about “The Monthly Recalculation Loop, the Distilled Tool and a Falsification Plan”?
Answer: Chapter 8 gives a monthly recalculation routine with an event rule for version changes, the distilled "Three-Light Cut-Point Calculator", a pre-registered test needing 1177 conversations per group, and the conditions that would overturn the book's three pillars. Boxes separate published evidence, runnable Python calculations (18 programs), classical sources and the author's hypotheses.
Scope of the numbers: Private customer service has a third action, a bot draft checked by a person, so it needs a six-cell loss matrix with two cut points; under illustrative parameters red lights below a chance of being right of 0.4000 and green from 0.9500. Parameters are illustrative, all conversations are fictional, and all numbers come from the book's programs, which read and write no file and connect to no customer-service system.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 1177
| arm X: mean 2.391 yuan per chat, sd 8.148 | ||
| arm Y: mean 3.451 yuan per chat, sd 10.107 | ||
| effect | delta/chat | n per arm |
| 1.00 | 1.060 | 1,177 |
| 0.50 | 0.530 | 4,707 |
| 0.25 | 0.265 | 18,826 |
| check at n = 1,177 per arm: simulated power 0.795 |
Show the full program
# Chapter 8. p82: how many chats a pre-registered A/B needs.
# Model (illustrative): arm X uses the three-light cuts of p21 (green
# from 0.95, red below 0.40); arm Y the accuracy-tuned single cut of
# p11 (answer when p >= 0.817, else a human). Per-chat cost is random:
# green costs 50 if wrong; amber costs 2, plus 50 if wrong and the
# reviewer misses it (rate 0.2); red costs 8. Means and variances are
# exact on the p11 bot. n per arm = 2 (z_a + z_b)^2 var / delta^2 with
# the pooled variance, two-sided alpha 0.05, power 0.8. A check runs
# 400 simulated trials (seed 20261113) of a two-sample z test.
import bisect
import math
import random
from statistics import NormalDist
LW, CA, M, CR, A2 = 50.0, 2.0, 0.8, 8.0, 0.817
ZA, ZB = NormalDist().inv_cdf(0.975), NormalDist().inv_cdf(0.8)
def nodes(mu=3.0, sd=1.6, n=4001):
zs = [mu + sd * (-8.0 + 16.0 * i / (n - 1)) for i in range(n)]
ws = [math.exp(-0.5 * ((z - mu) / sd) ** 2) for z in zs]
tot = math.fsum(ws)
return [(1.0 / (1.0 + math.exp(-z)), w / tot) for z, w in zip(zs, ws)]
def outcomes(arm, p):
# list of (cost, probability) for one chat with P(right) = p
if arm == "X":
if p >= 0.95:
return [(LW, 1 - p), (0.0, p)]
if p >= 0.40:
miss = (1 - p) * (1 - M)
return [(CA + LW, miss), (CA, 1 - miss)]
return [(CR, 1.0)]
return [(LW, 1 - p), (0.0, p)] if p >= A2 else [(CR, 1.0)]
PW = nodes()
stats = {}
for arm in ("X", "Y"):
m1 = math.fsum(w * q * c for p, w in PW for c, q in outcomes(arm, p))
m2 = math.fsum(w * q * c * c for p, w in PW
for c, q in outcomes(arm, p))
stats[arm] = (m1, m2 - m1 * m1)
print("arm %s: mean %.3f yuan per chat, sd %.3f"
% (arm, m1, math.sqrt(m2 - m1 * m1)))
var = (stats["X"][1] + stats["Y"][1]) / 2
full = stats["Y"][0] - stats["X"][0]
print(" effect delta/chat n per arm")
for frac in (1.0, 0.5, 0.25):
d = full * frac
n = math.ceil(2 * (ZA + ZB) ** 2 * var / (d * d))
print("%6.2f %10.3f %9s" % (frac, d, format(n, ",")))
n = math.ceil(2 * (ZA + ZB) ** 2 * var / (full * full))
rng, rej = random.Random(20261113), 0
ws = [w for _, w in PW]
cdf = [math.fsum(ws[:k + 1]) for k in range(len(ws))]
def one_chat(arm):
p = PW[min(len(PW) - 1, bisect.bisect_left(cdf, rng.random()))][0]
out = outcomes(arm, p)
return out[0][0] if rng.random() < out[0][1] else out[-1][0]
for _ in range(400):
xs = [one_chat("X") for _ in range(n)]
ys = [one_chat("Y") for _ in range(n)]
mx, my = math.fsum(xs) / n, math.fsum(ys) / n
vx = math.fsum((x - mx) ** 2 for x in xs) / (n - 1)
vy = math.fsum((y - my) ** 2 for y in ys) / (n - 1)
rej += abs(my - mx) / math.sqrt((vx + vy) / n) > ZA
print("check at n = %s per arm: simulated power %.3f"
% (format(n, ","), rej / 400))Single source and citation
Fan, Y. (2026). Three Lights for Handing Over between Bot and Human in Private Customer Service: Misclassification Loss Matrices, Chow's Reject Rule and Optimal Cut Points on Calibrated Confidence (Version 1.0.0), Chapter 8: The Monthly Recalculation Loop, the Distilled Tool and a Falsification Plan. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-266 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026266,
author = {Fan, Yafu},
title = {Three Lights for Handing Over between Bot and Human in Private Customer Service: Misclassification Loss Matrices, Chow's Reject Rule and Optimal Cut Points on Calibrated Confidence},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Blind Spot of Accuracy: Chow's Reject Rule and the Cost Ratio
- Chapter 2 · A Six-Cell Loss Matrix: Bayes Risk in Three Zones and Two Cut Points
- Chapter 3 · Cost Curves: Why the More Accurate Bot Can Cost More
- Chapter 4 · Confidence Is Not Probability: Isotonic Calibration and the Price of Few Labels
- Chapter 5 · Big Sales and Drift: Bayesian Correction for Prior Shift and the Sampling Error of Cut Points
- Chapter 6 · Red Lights Beyond Price: Stop Means Stop, Vulnerable Customers Go to Staff, and the Staffing Cap
- Chapter 7 · Setting the Doubtful Aside, and Bayes: Two Classical Texts on Being Unsure