What does The Explanatory-Depth Razor conclude, in numbers, about “Calibration, Not Confidence: the Brier Score, Murphy's Decomposition and the Statistical Artefact in the Dunning-Kruger Chart”?
Answer: Chapter 2 covers the Brier score and Murphy's decomposition, the real hit rate of ninety per cent ranges (0.589 when ranges are half as wide as they should be) and the statistical artefact in the quartile chart.
Scope of the numbers: Parameters are illustrative, all data are synthetic, and every calculation is generated by the book's programs. Chapter 1 measures, in synthetic data, a drop of 1.283 points in self-rated understanding of devices against an interval for facts from -0.263 to 0.063 that includes zero.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.589
| k = s_stated / s_true | exact hit | simulated hit | misses of 400 |
| 0.25 | 0.319 | 0.295 | 282 |
| 0.40 | 0.489 | 0.495 | 202 |
| 0.50 | 0.589 | 0.590 | 164 |
| 0.70 | 0.750 | 0.743 | 103 |
| 1.00 | 0.900 | 0.905 | 38 |
| 1.30 | 0.968 | 0.963 | 15 |
| observed hit 0.50 -> stated ranges are 0.410 of true sd; widen by 2.44 | |||
| observed hit 0.60 -> stated ranges are 0.512 of true sd; widen by 1.95 | |||
| observed hit 0.70 -> stated ranges are 0.630 of true sd; widen by 1.59 |
Show the full program
# Chapter 2. p22: how often do "90 per cent" ranges contain the truth?
# A judge states a 90% range m +- 1.645 s_stated for a quantity whose
# real error is normal with sd s_true. If s_stated = k s_true, the true
# hit rate is 2 Phi(1.645 k) - 1. We also simulate 400 estimates of
# monthly order counts (log scale) to show the sampling noise around it.
# Shrink factors are illustrative; Alpert & Raiffa (1982) and Lichtenstein
# et al. (1982) report hit rates far below the nominal level.
import math
import random
from statistics import NormalDist
Z = NormalDist()
Z90 = 1.644854
rng = random.Random(39622)
N = 400
print("k = s_stated / s_true exact hit simulated hit misses of 400")
for k in (0.25, 0.40, 0.50, 0.70, 1.00, 1.30):
exact = 2.0 * Z.cdf(Z90 * k) - 1.0
hits = 0
for _ in range(N):
err = rng.gauss(0.0, 1.0)
if abs(err) <= Z90 * k:
hits += 1
print(" {:4.2f} {:6.3f} {:6.3f} {:4d}"
.format(k, exact, hits / N, N - hits))
# widening needed: observed hit rate h implies k = z((1+h)/2) / 1.645
for h in (0.50, 0.60, 0.70):
k = Z.inv_cdf((1.0 + h) / 2.0) / Z90
print("observed hit {:.2f} -> stated ranges are {:.3f} of true sd;"
" widen by {:.2f}".format(h, k, 1.0 / k))Single source and citation
Fan, Y. (2026). Shattering the Illusion of Explanatory Depth, Epistemic Calibration and the Razor Separating True Knowledge from Assumed Familiarity (Version 1.0.0), Chapter 2: Calibration, Not Confidence: the Brier Score, Murphy's Decomposition and the Statistical Artefact in the Dunning-Kruger Chart. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-396 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026396,
author = {Fan, Yafu},
title = {Shattering the Illusion of Explanatory Depth, Epistemic Calibration and the Razor Separating True Knowledge from Assumed Familiarity},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Boundary of the Illusion of Explanatory Depth: Strongest for Mechanisms, Weakest for Facts
- Chapter 3 · The Four-Step Razor: Causal Chain, Order of Magnitude, Failure Boundary and Counterexample
- Chapter 4 · Scoring the Assumption List and the Value of Information: Which Assumption to Test First
- Chapter 5 · Reasons Against, the Premortem and Sealing the Ledger against Hindsight
- Chapter 6 · Turning "Fatal" and "Stripping" into Intervals: a Band of Decision Loss, a Small Effect and the Conditions for Refutation
- Chapter 7 · Reading the Classics: the Xunzi on Dispelling Obsession, Chapter Seventy-One of the Laozi and Descartes's Four Rules
- Chapter 8 · Field Tools and Checker Specifications: the False-Knowledge Register, the Four-Step Razor Scorer and a Pre-Registered Test