What does The Explanatory-Depth Razor conclude, in numbers, about “Calibration, Not Confidence: the Brier Score, Murphy's Decomposition and the Statistical Artefact in the Dunning-Kruger Chart”?

Answer: Chapter 2 covers the Brier score and Murphy's decomposition, the real hit rate of ninety per cent ranges (0.589 when ranges are half as wide as they should be) and the statistical artefact in the quartile chart.

Scope of the numbers: Parameters are illustrative, all data are synthetic, and every calculation is generated by the book's programs. Chapter 1 measures, in synthetic data, a drop of 1.283 points in self-rated understanding of devices against an interval for facts from -0.263 to 0.063 that includes zero.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.589

k = s_stated / s_trueexact hitsimulated hitmisses of 400
0.250.3190.295282
0.400.4890.495202
0.500.5890.590164
0.700.7500.743103
1.000.9000.90538
1.300.9680.96315
observed hit 0.50 -> stated ranges are 0.410 of true sd; widen by 2.44
observed hit 0.60 -> stated ranges are 0.512 of true sd; widen by 1.95
observed hit 0.70 -> stated ranges are 0.630 of true sd; widen by 1.59
Show the full program
# Chapter 2. p22: how often do "90 per cent" ranges contain the truth?
# A judge states a 90% range m +- 1.645 s_stated for a quantity whose
# real error is normal with sd s_true. If s_stated = k s_true, the true
# hit rate is 2 Phi(1.645 k) - 1. We also simulate 400 estimates of
# monthly order counts (log scale) to show the sampling noise around it.
# Shrink factors are illustrative; Alpert & Raiffa (1982) and Lichtenstein
# et al. (1982) report hit rates far below the nominal level.
import math
import random
from statistics import NormalDist

Z = NormalDist()
Z90 = 1.644854
rng = random.Random(39622)
N = 400
print("k = s_stated / s_true   exact hit   simulated hit   misses of 400")
for k in (0.25, 0.40, 0.50, 0.70, 1.00, 1.30):
    exact = 2.0 * Z.cdf(Z90 * k) - 1.0
    hits = 0
    for _ in range(N):
        err = rng.gauss(0.0, 1.0)
        if abs(err) <= Z90 * k:
            hits += 1
    print("        {:4.2f}            {:6.3f}      {:6.3f}         {:4d}"
          .format(k, exact, hits / N, N - hits))
# widening needed: observed hit rate h implies k = z((1+h)/2) / 1.645
for h in (0.50, 0.60, 0.70):
    k = Z.inv_cdf((1.0 + h) / 2.0) / Z90
    print("observed hit {:.2f} -> stated ranges are {:.3f} of true sd;"
          " widen by {:.2f}".format(h, k, 1.0 / k))

Single source and citation

Fan, Y. (2026). Shattering the Illusion of Explanatory Depth, Epistemic Calibration and the Razor Separating True Knowledge from Assumed Familiarity (Version 1.0.0), Chapter 2: Calibration, Not Confidence: the Brier Score, Murphy's Decomposition and the Statistical Artefact in the Dunning-Kruger Chart. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-396 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026396,
  author    = {Fan, Yafu},
  title     = {Shattering the Illusion of Explanatory Depth, Epistemic Calibration and the Razor Separating True Knowledge from Assumed Familiarity},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book