关于「严格恰当评分规则:布莱尔分数与墨菲分解」,《概率校准与群体智慧》给出的定量结论是什么?

结论:布莱尔分数是预报与结果之差的平方的平均,墨菲把它分解为可靠性(校准)、区分度与不确定性三项;当组内预报相同时分解是精确的,分箱后则多出组内方差与协方差两项。总是报 50% 的人只在基础率为 50% 时校准完美;一般而言,总是报基础率的人校准完美而区分度为零。过度自信者可以拥有和校准者同样的区分度,却因为说得太满而得分更差。布莱尔分数与对数分数是严格恰当的,绝对误差不是。用很少的预报给个人算校准误差,会系统地偏高。下一章讨论概率判断中最著名的一种错误:忽略基础率。

数据口径:本书不是心理咨询、医学或投资建议。 书中以标注框区分已发表的证据、可运行的 Python 计算(16 段程序,全部核对)、原典与作者的假说。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 50

forecaster        |   BS    REL    RES    UNC
always 50 %       | 0.2500 0.0403 0.0000 0.2097
always base rate  | 0.2097 0.0000 0.0000 0.2097
calibrated        | 0.1677 0.0001 0.0413 0.2097
overconfident     | 0.1797 0.0115 0.0407 0.2097
underconfident    | 0.1757 0.0075 0.0394 0.2097
overconfident: mean forecast -> observed frequency (bins with n >= 200):
  0.02->0.09  0.09->0.25  0.20->0.33  0.30->0.37  0.40->0.44  0.50->0.49
  0.60->0.55  0.70->0.59  0.80->0.67  0.90->0.74  0.97->0.88
perfectly calibrated, only 20 forecasts: REL mean 0.0631, median 0.0583 (truth 0)

核心公式

\[P = \frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{r}\left(f_{ij} - o_{ij}\right)^2\]
查看完整验算程序
# Five forecasters, 20,000 events with base rate 30 %: always 50 %, always 30 %,
# calibrated (states the true chance), overconfident (2 x logit), underconfident (0.5 x
# logit about 30 %). Last part: REL from only 20 forecasts of a perfectly calibrated
# person.
import math, random

rng = random.Random(1973)

def logit(p):
    return math.log(p / (1 - p))

def inv(x):
    return 1 / (1 + math.exp(-x))

def decompose(f, o, nb=11):
    n, obar = len(f), sum(o) / len(o)
    bins = {}
    for fi, oi in zip(f, o):
        bins.setdefault(round(fi * (nb - 1)), []).append((fi, oi))
    rel = res = 0.0
    table = []
    for k in sorted(bins):
        pts = bins[k]
        fk = sum(p[0] for p in pts) / len(pts)
        ok = sum(p[1] for p in pts) / len(pts)
        rel += len(pts) * (fk - ok) ** 2
        res += len(pts) * (ok - obar) ** 2
        table.append((fk, ok, len(pts)))
    bs = sum((a - b) ** 2 for a, b in zip(f, o)) / n
    return bs, rel / n, res / n, obar * (1 - obar), table

def cases(n=20000):
    ps = [min(max(rng.betavariate(1.2, 2.8), 0.005), 0.995) for _ in range(n)]
    o = [1 if rng.random() < p else 0 for p in ps]
    fc = {"always 50 %": [0.5] * n,
          "always base rate": [0.3] * n,
          "calibrated": ps,
          "overconfident": [inv(2 * logit(p)) for p in ps],
          "underconfident": [inv(0.5 * logit(p) + 0.5 * logit(0.3)) for p in ps]}
    return {k: decompose(v, o) for k, v in fc.items()}

def small_sample_rel(m=20, reps=4000):
    vals = []
    for _ in range(reps):
        ps = [rng.betavariate(1.2, 2.8) for _ in range(m)]
        f = [round(p * 10) / 10 for p in ps]                 # states its chance, rounded
        o = [1 if rng.random() < q else 0 for q in f]        # events happen at that rate
        vals.append(decompose(f, o)[1])
    vals.sort()
    return sum(vals) / reps, vals[reps // 2]

if __name__ == "__main__":
    res = cases()
    print("forecaster        |   BS    REL    RES    UNC")
    for k, (bs, rel, rs, unc, _) in res.items():
        print(f"{k:17s} | {bs:.4f} {rel:.4f} {rs:.4f} {unc:.4f}")
    print("overconfident: mean forecast -> observed frequency (bins with n >= 200):")
    cells = [f"{f:.2f}->{ob:.2f}" for f, ob, n in res["overconfident"][4] if n >= 200]
    print("  " + "  ".join(cells[:6]))
    print("  " + "  ".join(cells[6:]))
    mean, med = small_sample_rel()
    print(f"perfectly calibrated, only 20 forecasts: REL mean {mean:.4f},"
          f" median {med:.4f} (truth 0)")

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《概率判断、布莱尔分数校准与群体智慧的统计理论》第 2 章「严格恰当评分规则:布莱尔分数与墨菲分解」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23764998

DOI: 10.5281/zenodo.23764998 · 阅读本章所在专著 · 5DL-B-026 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026026,
  author    = {Fan, Yafu},
  title     = {Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.23764998},
  url       = {https://doi.org/10.5281/zenodo.23764998}
}

本书其他章节结论