Shattering the Illusion of Explanatory Depth, Epistemic Calibration and the Razor Separating True Knowledge from Assumed Familiarity: chapter answers
Two sentences circulate in business conversation. One says that the deadliest thing is "thinking you know", and that writing out the mechanism makes false knowledge fall away; the other says that the less people know, the more confident they are. This book turns the first into claims that data can refute and removes the second from the decision maker's toolkit. Checking the sources forced four corrections. The illusion of explanatory depth is strongest for causal mechanisms in devices and natural phenomena and weak or absent for facts, procedures and narratives, so the razor is reserved for m…
DOI pending · monograph page goes live after publication · 5DL-B-396 · 中文版
- Chapter 1 · The Boundary of the Illusion of Explanatory Depth: Strongest for Mechanisms, Weakest for Facts
Chapter 1 measures, in synthetic data, a drop of 1.283 points in self-rated understanding of devices against an interval for facts from -0.263 to 0.063 that includes zero. - Chapter 2 · Calibration, Not Confidence: the Brier Score, Murphy's Decomposition and the Statistical Artefact in the Dunning-Kruger Chart
Chapter 2 covers the Brier score and Murphy's decomposition, the real hit rate of ninety per cent ranges (0.589 when ranges are half as wide as they should be) and the statistical artefact in the quartile chart. - Chapter 3 · The Four-Step Razor: Causal Chain, Order of Magnitude, Failure Boundary and Counterexample
The four-step razor scores every claim on four components, chain, magnitude, boundary and counterexample, and a weighted total of 0.60 passes. In the synthetic data the two claims held with the most confidence are exact… - Chapter 4 · Scoring the Assumption List and the Value of Information: Which Assumption to Test First
Now the value of information. At a prior of 0.400 the current decision is to hold; EVPI is 24.00 and the pilot's EVSI is 8.40, about 0.350 of the value of perfect information. At a prior of 0.550 EVSI rises to 19.05. Ex… - Chapter 5 · Reasons Against, the Premortem and Sealing the Ledger against Hindsight
Writing a reason against can shrink overconfidence, but it can improve only a thin layer of the Brier score; in this chapter's illustrative model, at most 0.00931. A premortem's coverage of failure causes saturates with… - Chapter 6 · Turning "Fatal" and "Stripping" into Intervals: a Band of Decision Loss, a Small Effect and the Conditions for Refutation
Chapter 6 samples wide illustrative parameters: routing between razor and retrieval cuts net loss by a median of 22.7 %, with 8.2 % at the fifth percentile and 41.2 % at the ninety-fifth, against a median of 5.1 % when… - Chapter 7 · Reading the Classics: the Xunzi on Dispelling Obsession, Chapter Seventy-One of the Laozi and Descartes's Four Rules
The true overall repeat rate is 0.3250. The reader of one corner has an error of 0.2494 with a sample of 20, and with a sample of 1,280 the error has fallen only to 0.2244; it barely moves. The stratified reader's error… - Chapter 8 · Field Tools and Checker Specifications: the False-Knowledge Register, the Four-Step Razor Scorer and a Pre-Registered Test
Chapter 8 sets out field tools and checker specifications: the false-knowledge register, the distilled Four-Step Razor and Pre-Decision Assumption-List Scorer, checkers grouped by kit, and a pre-registered test with its…
Illustrative model calculation, not a live quote