What does Human Go Decisions after AI conclude, in numbers, about “The Limits of the Yardstick: Evaluator Noise, Engine Versions and the Win-Rate Ceiling”?

Answer: Chapter 6 tests the yardstick: evaluator noise averages out in large samples, and the same 3-point mistake costs 14.566 points of win rate in an even position and 1.431 when far ahead.

Scope of the numbers: Apart from the paper's figures, all parameters are illustrative and all calculations are generated by the book's programs; the book is non-personalised business guidance and addresses no individual decision. Chapter 2 demonstrates regression discontinuity in time on simulated data: with a true jump of 0.30 a 12-month bandwidth detects it only 0.685 of the time, while a world with nothing but a ramp is declared significant with probability 0.258 at 36 months.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 1.431, 14.566

part 1: win-rate cost of a mistake, by score lead
lead win rate cost 1 pt cost 3 pts cost 10 pts
-200.018 0.321 0.803 1.551
-100.119 1.945 5.006 10.122
-50.269 3.747 10.096 22.152
+00.500 4.983 14.566 38.080
+50.731 4.108 13.237 46.212
+100.881 2.265 7.861 38.080
+200.982 0.390 1.431 10.122
part 2: average cost of a 3-point mistake
close games (sd 5)12.177
lopsided games (sd 12)8.031
apparent DQI gain with no better play: +4.146
score loss of the same move3 points in both cases
Show the full program
# Chapter 6. p62: the same mistake costs different win rate in
# different positions, so DQI also measures the state of the game
# Model (illustrative): the evaluator maps a score lead x (points) to
# a win rate 1 / (1 + exp(-x / s)) with s = 5. A move that gives away
# c points costs 100 x [w(x) - w(x - c)] points of win rate. Part 2
# averages the cost of a 3-point mistake over leads drawn from a
# normal law with sd 5 (close games) or sd 12 (lopsided games), by a
# fine grid and math.fsum. A fall in average win-rate loss can thus
# come from more lopsided games instead of better moves.
import math

SCALE = 5.0


def wr(x):
    return 1.0 / (1.0 + math.exp(-x / SCALE))


def cost(x, c):
    return 100.0 * (wr(x) - wr(x - c))


def avg_cost(c, sd):
    step = 0.05
    xs = [-80.0 + step * i for i in range(int(160.0 / step) + 1)]
    dens = [math.exp(-0.5 * (x / sd) ** 2) for x in xs]
    tot = math.fsum(dens)
    return math.fsum(d * cost(x, c) for d, x in zip(dens, xs)) / tot


if __name__ == "__main__":
    print("part 1: win-rate cost of a mistake, by score lead")
    print("  lead    win rate   cost 1 pt   cost 3 pts   cost 10 pts")
    for x in (-20, -10, -5, 0, 5, 10, 20):
        print("  {:+4d}     {:.3f}      {:6.3f}      {:6.3f}       {:6.3f}"
              .format(x, wr(x), cost(x, 1), cost(x, 3), cost(x, 10)))
    print("part 2: average cost of a 3-point mistake")
    close = avg_cost(3.0, 5.0)
    wide = avg_cost(3.0, 12.0)
    print("  close games (sd 5):     {:.3f}".format(close))
    print("  lopsided games (sd 12): {:.3f}".format(wide))
    print("  apparent DQI gain with no better play: {:+.3f}".format(
        close - wide))
    print("score loss of the same move: 3 points in both cases")

Single source and citation

Fan, Y. (2026). Post-AI Leaps in Human Go Decision Quality, Novel-Move Rates and Strategic Entropy in the Thirteen-Chapter Classic of Go (Version 1.0.0), Chapter 6: The Limits of the Yardstick: Evaluator Noise, Engine Versions and the Win-Rate Ceiling. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-230 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026230,
  author    = {Fan, Yafu},
  title     = {Post-AI Leaps in Human Go Decision Quality, Novel-Move Rates and Strategic Entropy in the Thirteen-Chapter Classic of Go},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book