What does Human Go Decisions after AI conclude, in numbers, about “The Limits of the Yardstick: Evaluator Noise, Engine Versions and the Win-Rate Ceiling”?
Answer: Chapter 6 tests the yardstick: evaluator noise averages out in large samples, and the same 3-point mistake costs 14.566 points of win rate in an even position and 1.431 when far ahead.
Scope of the numbers: Apart from the paper's figures, all parameters are illustrative and all calculations are generated by the book's programs; the book is non-personalised business guidance and addresses no individual decision. Chapter 2 demonstrates regression discontinuity in time on simulated data: with a true jump of 0.30 a 12-month bandwidth detects it only 0.685 of the time, while a world with nothing but a ramp is declared significant with probability 0.258 at 36 months.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 1.431, 14.566
| part 1: win-rate cost of a mistake, by score lead | |
| lead win rate cost 1 pt cost 3 pts cost 10 pts | |
| -20 | 0.018 0.321 0.803 1.551 |
| -10 | 0.119 1.945 5.006 10.122 |
| -5 | 0.269 3.747 10.096 22.152 |
| +0 | 0.500 4.983 14.566 38.080 |
| +5 | 0.731 4.108 13.237 46.212 |
| +10 | 0.881 2.265 7.861 38.080 |
| +20 | 0.982 0.390 1.431 10.122 |
| part 2: average cost of a 3-point mistake | |
| close games (sd 5) | 12.177 |
| lopsided games (sd 12) | 8.031 |
| apparent DQI gain with no better play: +4.146 | |
| score loss of the same move | 3 points in both cases |
Show the full program
# Chapter 6. p62: the same mistake costs different win rate in
# different positions, so DQI also measures the state of the game
# Model (illustrative): the evaluator maps a score lead x (points) to
# a win rate 1 / (1 + exp(-x / s)) with s = 5. A move that gives away
# c points costs 100 x [w(x) - w(x - c)] points of win rate. Part 2
# averages the cost of a 3-point mistake over leads drawn from a
# normal law with sd 5 (close games) or sd 12 (lopsided games), by a
# fine grid and math.fsum. A fall in average win-rate loss can thus
# come from more lopsided games instead of better moves.
import math
SCALE = 5.0
def wr(x):
return 1.0 / (1.0 + math.exp(-x / SCALE))
def cost(x, c):
return 100.0 * (wr(x) - wr(x - c))
def avg_cost(c, sd):
step = 0.05
xs = [-80.0 + step * i for i in range(int(160.0 / step) + 1)]
dens = [math.exp(-0.5 * (x / sd) ** 2) for x in xs]
tot = math.fsum(dens)
return math.fsum(d * cost(x, c) for d, x in zip(dens, xs)) / tot
if __name__ == "__main__":
print("part 1: win-rate cost of a mistake, by score lead")
print(" lead win rate cost 1 pt cost 3 pts cost 10 pts")
for x in (-20, -10, -5, 0, 5, 10, 20):
print(" {:+4d} {:.3f} {:6.3f} {:6.3f} {:6.3f}"
.format(x, wr(x), cost(x, 1), cost(x, 3), cost(x, 10)))
print("part 2: average cost of a 3-point mistake")
close = avg_cost(3.0, 5.0)
wide = avg_cost(3.0, 12.0)
print(" close games (sd 5): {:.3f}".format(close))
print(" lopsided games (sd 12): {:.3f}".format(wide))
print(" apparent DQI gain with no better play: {:+.3f}".format(
close - wide))
print("score loss of the same move: 3 points in both cases")Single source and citation
Fan, Y. (2026). Post-AI Leaps in Human Go Decision Quality, Novel-Move Rates and Strategic Entropy in the Thirteen-Chapter Classic of Go (Version 1.0.0), Chapter 6: The Limits of the Yardstick: Evaluator Noise, Engine Versions and the Win-Rate Ceiling. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-230 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026230,
author = {Fan, Yafu},
title = {Post-AI Leaps in Human Go Decision Quality, Novel-Move Rates and Strategic Entropy in the Thirteen-Chapter Classic of Go},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Leap at the Break: a Decision Quality Index on a KataGo Yardstick
- Chapter 2 · A Break in Time: Regression Discontinuity at March 2016
- Chapter 3 · Novel-Move Rates: the First Historically New Move, a Growing Database and the Memorisation Test
- Chapter 4 · Strategic Entropy: Joseki Concentration, Local Optima and Conformist Lock-In
- Chapter 5 · The Contest over Mechanism: Internalisation, Memorisation and Composition
- Chapter 7 · Classics East and West: the Classic of Go in Thirteen Chapters and Franklin's The Morals of Chess
- Chapter 8 · Closing the Loop: a Team Routine for AI Reviews and an Audit of Editorial Decision Quality