5D Library › Books › 5DL-B-026
概率判断、布莱尔分数校准与群体智慧的统计理论
从基础率忽略、信号检测论到超级预测者的贝叶斯更新及古典兵家庙算观考辨
Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence: From Base-Rate Neglect and Signal Detection Theory to Superforecasting, Read Beside Classical Chinese Strategic Calculation
DOI10.5281/zenodo.23764998摘要
「大概会成」到底是多大的可能？说这句话的人这些年说对了多少？本书用判断与决策心理学、预测评估与统计学考察怎样测量概率判断的准确性，判断会以哪些系统性的方式出错，以及把许多人的判断合在一起何时更准。依次讨论：泰洛克对 284 名专家约二十年预测的研究（约 28,000 次预测，另有书评引用的 82,361 次计数；专家平均略好于随机猜测、不如简单外推，狐狸型好于刺猬型，「不如掷飞镖的黑猩猩」是通俗说法），预测锦标赛中的训练、组队与超级预测者，以及过度自信与反应不足、反应过度的模型；布莱尔分数与墨菲分解（可靠性 − 区分度 + 不确定性，分箱时多出组内两项），「总是报基础率者校准完美而区分度为零」（「总是报 50%」只在基础率为 50% 时校准），以及严格恰当评分规则的推导；基础率忽略、患病率 1% 时阳性预测值约 16.1% 与自然频率（元分析中正确率约从 4% 到 24%）；信号检测论的 d′、判断标准与代价不对称下的最优似然比门槛；计划谬误、参考类别预测（258 个交通项目近九成超支，铁路平均 44.7%）与经验贝叶斯收缩；高尔顿称牛重的中位数与平均数之更正、多样性预测定理（恒等式而非经验规律）、相关误差使群体无法胜过专家的条件，以及平均概率为何需要推向极端。
八章与《孙子兵法·计篇》的「庙算」与五事七计、《鬼谷子·揣篇》的「量权」（更正：不是独立篇名，原文为「何谓量权？曰：度于大小，谋于众寡」）及《韩非子·显学》「无参验而必之者，愚也」并读，并明确古代确定性断语与现代概率校准的界限；最后给出一份四步概率校准核查表，以及一项比较 15 分钟校准微训练（自然频率转换 + 外部参考类别锚定 + 布莱尔分数与区间命中反馈）与时长匹配的逻辑思维阅读的预注册随机对照试验方案（唯一主要结局为 20 道题上 80% 区间的校准误差 |H − 0.80|，每组 133 人可分析、招募 148 人，共 296 人，判定规则写成代码）。书中以标注框区分已发表的证据、可运行的 Python 计算（16 段程序，全部核对）、原典与作者的假说。本书不是心理咨询、医学或投资建议。
关键词：概率判断；校准；布莱尔分数；墨菲分解；恰当评分规则；对数分数；过度自信；专家政治判断；超级预测者；贝叶斯更新；基础率忽略；自然频率；信号检测论；ROC 曲线；计划谬误；参考类别预测；经验贝叶斯；群体智慧；多样性预测定理；孙子兵法
Abstract
How likely, exactly, is "it'll probably work out"? How often has the speaker been right? This book uses the psychology of judgement and decision making, forecast verification and statistics to ask how the accuracy of probabilistic judgements can be measured, in what systematic ways they go wrong, and when combining many people's judgements makes them more accurate. It covers Tetlock's study of 284 experts' forecasts over about two decades (some 28,000 forecasts, with a count of 82,361 also quoted by reviewers; experts did slightly better than random guessing but worse than simple extrapolation, foxes beat hedgehogs, and "worse than a dart-throwing chimp" is a popular paraphrase), training, teaming and superforecasters in forecasting tournaments, and models of overconfidence, under-reaction and over-reaction; the Brier score and Murphy's decomposition (reliability − resolution + uncertainty, with two within-bin terms after binning), the fact that always stating the base rate is perfectly calibrated with zero resolution (always saying 50 % is calibrated only when the base rate is 50 %), and the derivation of strictly proper scoring rules; base-rate neglect, a positive predictive value of about 16.1 % at 1 % prevalence, and natural frequencies (in a meta-analysis accuracy rose from about 4 % to 24 %); signal detection theory's d′, criterion and optimal likelihood-ratio threshold under unequal costs; the planning fallacy, reference-class forecasting (almost nine in ten of 258 transport projects overran, rail by 44.7 % on average) and empirical-Bayes shrinkage; and the corrected median and mean of Galton's ox-weighing, the diversity prediction theorem (an identity, not an empirical law), the conditions under which correlated errors stop a crowd from beating an expert, and why averaged probabilities should be extremised.
The eight chapters are read beside the "temple calculation" and the five factors and seven assessments of the Sunzi's chapter "Estimates", the "weighing of power" in the Guiguzi's chapter "Assessing" (corrected: not a chapter title; the text reads "What is weighing power? It is to measure great and small, and to reckon many and few"), and the Han Feizi's "to be certain without cross-checking is folly", stating clearly the boundary between categorical classical pronouncements and modern probability calibration. They end with a four-step probability-calibration checklist and a pre-registered randomised controlled trial comparing a 15-minute calibration micro-training (natural-frequency conversion, anchoring on an outside reference class, Brier-score and interval-hit feedback) with time-matched reading on logical thinking (single primary outcome the calibration error |H − 0.80| of 80 % intervals on 20 items, 133 analysable and 148 recruited participants per arm, 296 in all, decision rule written as code). Marked boxes separate published evidence, runnable Python calculations (16 programs, all checked), classical sources and the author's hypotheses. This book is not counselling, medical or investment advice.
Keywords: probabilistic judgement; calibration; Brier score; Murphy decomposition; proper scoring rules; logarithmic score; overconfidence; expert political judgement; superforecasting; Bayesian updating; base-rate neglect; natural frequencies; signal detection theory; ROC curve; planning fallacy; reference-class forecasting; empirical Bayes; wisdom of crowds; diversity prediction theorem; Sunzi
目录 · Contents
- 序：把「大概」变成数字English 中文Preface: Turning "Probably" into a Number
- 第一章 泰洛克的预测研究：专家、狐狸与超级预测者English 中文Chapter 1 Tetlock's Forecasting Research: Experts, Foxes and Superforecasters
- 第二章 严格恰当评分规则：布莱尔分数与墨菲分解English 中文Chapter 2 Strictly Proper Scoring Rules: The Brier Score and Murphy's Decomposition
- 第三章 基础率忽略与自然频率English 中文Chapter 3 Base-Rate Neglect and Natural Frequencies
- 第四章 信号检测论：看得清与门槛放在哪里English 中文Chapter 4 Signal Detection Theory: Seeing Clearly and Where to Set the Threshold
- 第五章 参考类别预测与计划谬误English 中文Chapter 5 Reference-Class Forecasting and the Planning Fallacy
- 第六章 群体智慧的统计理论English 中文Chapter 6 The Statistics of Collective Intelligence
- 第七章 原典并读：庙算、量权与参验English 中文Chapter 7 Reading the Classics: Temple Calculation, Weighing Power and Cross-Checking
- 第八章 实践与预注册试验English 中文Chapter 8 Practice and a Pre-registered Trial
How to cite · 引用
Fan, Y. (2026). Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence: From Base-Rate Neglect and Signal Detection Theory to Superforecasting, Read Beside Classical Chinese Strategic Calculation [概率判断、布莱尔分数校准与群体智慧的统计理论] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23764998
BibTeX
@book{fan2026judgment,
title = {{Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence}: From Base-Rate Neglect and Signal Detection Theory to Superforecasting, Read Beside Classical Chinese Strategic Calculation},
author = {Fan, Yafu},
year = {2026},
publisher = {5D Library},
edition = {Version 1.0.0},
doi = {10.5281/zenodo.23764998},
url = {https://doi.org/10.5281/zenodo.23764998},
note = {Chinese title: 概率判断、布莱尔分数校准与群体智慧的统计理论}
}
| DOI | https://doi.org/10.5281/zenodo.23764998 |
|---|---|
| ORCID | https://orcid.org/0009-0004-3519-1203 |
| 5D Library ID | 5DL-B-026 |
| 版本 Version | 1.0.0 · 中英双语版 Bilingual edition |
| 出版日期 Date | 2026-10-29 |
| 网址 Online | https://5dlibrary.com/books/judgment-calibration-forecasting/ |
| SHA-256 · judgment-calibration-forecasting_bilingual_v1.0.0.pdf | f40e6a6f9392ebd2fd1096f68b84136d131978f8bbafafc3a04c950b4a0e59f7 |
| SHA-256 · judgment-calibration-forecasting_en_v1.0.0.pdf | f9890c99aa03107fa090b018700edf2eac357eacb00a9e6d022e9b9ba00687ee |
| SHA-256 · judgment-calibration-forecasting_zh_v1.0.0.pdf | 9f78b123fc271e886c017baf449df9a256d1b02a0e63b211d321672f3718a35b |
| Licence · 许可 | Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0). https://creativecommons.org/licenses/by-nc-nd/4.0/ |
Disclosure. The author uses the works of 5D Library, including this book, as the methodological basis of his community courses, the 5D Ascension Cards (Dimensional Shift Card Decks) and the Yafu AI knowledge base, and therefore has a commercial interest in the subject of this book. The book promotes no course, card deck or service; the Chapter 8 trial is designed to be run and analysed by researchers independent of the author's business and does not recruit from the author's community, students or customers; all the code and the study protocol are published openly for anyone to check. The author holds no financial-services licence and provides no investment advice, account management or trading-signal service.