关于「严格恰当评分规则:布莱尔分数与墨菲分解」,《概率校准与群体智慧》给出的定量结论是什么?
结论:布莱尔分数是预报与结果之差的平方的平均,墨菲把它分解为可靠性(校准)、区分度与不确定性三项;当组内预报相同时分解是精确的,分箱后则多出组内方差与协方差两项。总是报 50% 的人只在基础率为 50% 时校准完美;一般而言,总是报基础率的人校准完美而区分度为零。过度自信者可以拥有和校准者同样的区分度,却因为说得太满而得分更差。布莱尔分数与对数分数是严格恰当的,绝对误差不是。用很少的预报给个人算校准误差,会系统地偏高。下一章讨论概率判断中最著名的一种错误:忽略基础率。
数据口径:本书不是心理咨询、医学或投资建议。 书中以标注框区分已发表的证据、可运行的 Python 计算(16 段程序,全部核对)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 50
forecaster | BS REL RES UNC always 50 % | 0.2500 0.0403 0.0000 0.2097 always base rate | 0.2097 0.0000 0.0000 0.2097 calibrated | 0.1677 0.0001 0.0413 0.2097 overconfident | 0.1797 0.0115 0.0407 0.2097 underconfident | 0.1757 0.0075 0.0394 0.2097 overconfident: mean forecast -> observed frequency (bins with n >= 200): 0.02->0.09 0.09->0.25 0.20->0.33 0.30->0.37 0.40->0.44 0.50->0.49 0.60->0.55 0.70->0.59 0.80->0.67 0.90->0.74 0.97->0.88 perfectly calibrated, only 20 forecasts: REL mean 0.0631, median 0.0583 (truth 0)
核心公式
\[P = \frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{r}\left(f_{ij} - o_{ij}\right)^2\]
查看完整验算程序
# Five forecasters, 20,000 events with base rate 30 %: always 50 %, always 30 %,
# calibrated (states the true chance), overconfident (2 x logit), underconfident (0.5 x
# logit about 30 %). Last part: REL from only 20 forecasts of a perfectly calibrated
# person.
import math, random
rng = random.Random(1973)
def logit(p):
return math.log(p / (1 - p))
def inv(x):
return 1 / (1 + math.exp(-x))
def decompose(f, o, nb=11):
n, obar = len(f), sum(o) / len(o)
bins = {}
for fi, oi in zip(f, o):
bins.setdefault(round(fi * (nb - 1)), []).append((fi, oi))
rel = res = 0.0
table = []
for k in sorted(bins):
pts = bins[k]
fk = sum(p[0] for p in pts) / len(pts)
ok = sum(p[1] for p in pts) / len(pts)
rel += len(pts) * (fk - ok) ** 2
res += len(pts) * (ok - obar) ** 2
table.append((fk, ok, len(pts)))
bs = sum((a - b) ** 2 for a, b in zip(f, o)) / n
return bs, rel / n, res / n, obar * (1 - obar), table
def cases(n=20000):
ps = [min(max(rng.betavariate(1.2, 2.8), 0.005), 0.995) for _ in range(n)]
o = [1 if rng.random() < p else 0 for p in ps]
fc = {"always 50 %": [0.5] * n,
"always base rate": [0.3] * n,
"calibrated": ps,
"overconfident": [inv(2 * logit(p)) for p in ps],
"underconfident": [inv(0.5 * logit(p) + 0.5 * logit(0.3)) for p in ps]}
return {k: decompose(v, o) for k, v in fc.items()}
def small_sample_rel(m=20, reps=4000):
vals = []
for _ in range(reps):
ps = [rng.betavariate(1.2, 2.8) for _ in range(m)]
f = [round(p * 10) / 10 for p in ps] # states its chance, rounded
o = [1 if rng.random() < q else 0 for q in f] # events happen at that rate
vals.append(decompose(f, o)[1])
vals.sort()
return sum(vals) / reps, vals[reps // 2]
if __name__ == "__main__":
res = cases()
print("forecaster | BS REL RES UNC")
for k, (bs, rel, rs, unc, _) in res.items():
print(f"{k:17s} | {bs:.4f} {rel:.4f} {rs:.4f} {unc:.4f}")
print("overconfident: mean forecast -> observed frequency (bins with n >= 200):")
cells = [f"{f:.2f}->{ob:.2f}" for f, ob, n in res["overconfident"][4] if n >= 200]
print(" " + " ".join(cells[:6]))
print(" " + " ".join(cells[6:]))
mean, med = small_sample_rel()
print(f"perfectly calibrated, only 20 forecasts: REL mean {mean:.4f},"
f" median {med:.4f} (truth 0)")唯一数据源与引用
范雅福(Fan, Y.). (2026). 《概率判断、布莱尔分数校准与群体智慧的统计理论》第 2 章「严格恰当评分规则:布莱尔分数与墨菲分解」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23764998
DOI: 10.5281/zenodo.23764998 · 阅读本章所在专著 · 5DL-B-026 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026026,
author = {Fan, Yafu},
title = {Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.23764998},
url = {https://doi.org/10.5281/zenodo.23764998}
}