关于「实践与预注册试验」,《概率校准与群体智慧》给出的定量结论是什么?

结论:四步校准核查表把前几章的结论落到四个可记录的环节:用数字替代模糊词,先找外部参考类别的基础率,事前写下更新触发条件,季度结束后用布莱尔分数复盘并与基础率比较;虚构示例显示,直觉数字可能比只报基础率还差,而按事先约定更新后有所改善,但一个季度的 12 个问题不足以下结论。试验方案比较 15 分钟校准微训练与时长匹配的逻辑思维阅读,唯一的主要结局是 20 道题上 80% 区间的校准误差 \(|H - 0.80|\),每组 133 人可分析、招募 148 人,判定规则包括操作检查与排除比例的门槛,写成代码并事先公开;无论结果正负都发表。

数据口径:本书不是心理咨询、医学或投资建议。 书中以标注框区分已发表的证据、可运行的 Python 计算(16 段程序,全部核对)、原典与作者的假说。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.80, 133, 148, 20

simulated control: mean CE 0.304, SD 0.173; hit chance 0.57 gives mean CE 0.243
analysable participants per arm | rho0.0 | 0.3 | 0.5
CE difference 0.05 (d0.29) | 253 | 231 | 190
CE difference 0.06 (d0.35) | 176 | 161 | 133
CE difference 0.08 (d0.46) | 100 | 91 | 75
chosen0.06, rho = 0.5 -> 133 per arm
allowing 10 % loss (incomplete sessions, failed attention checks):
recruit 148 per arm, 296 in all
check without covariate176 per arm, simulated power 0.93
floor: a perfectly calibrated person (h0.80) has expected CE 0.070 on 20 items
查看完整验算程序
# Sample size. Primary outcome CE = |H - 0.80| over 20 items (H = share of 80 %
# intervals that hit). ANCOVA on pre-test CE (correlation rho); alpha 0.05 two-sided,
# power 0.90. SD of CE from a simulated control arm: hit chance ~ Beta(mean 0.50, SD
# 0.15), an assumption.
import math, random
from statistics import NormalDist, mean, stdev

rng = random.Random(15)
N = NormalDist()

def beta_ab(m, sd):
    k = m * (1 - m) / sd ** 2 - 1
    return m * k, (1 - m) * k

def ce_sample(n, m, sd=0.15, items=20):
    a, b = beta_ab(m, sd)
    out = []
    for _ in range(n):
        h = rng.betavariate(a, b)
        hits = sum(rng.random() < h for _ in range(items))
        out.append(abs(hits / items - 0.80))
    return out

def per_arm(delta, sigma, rho, alpha=0.05, power=0.90):
    z = N.inv_cdf(1 - alpha / 2) + N.inv_cdf(power)
    return math.ceil(2 * z * z * sigma ** 2 * (1 - rho ** 2) / delta ** 2) + 1

if __name__ == "__main__":
    ctrl, trt = ce_sample(20000, 0.50), ce_sample(20000, 0.57)
    sigma = stdev(ctrl)
    print(f"simulated control: mean CE {mean(ctrl):.3f}, SD {sigma:.3f};"
          f" hit chance 0.57 gives mean CE {mean(trt):.3f}")
    print("analysable participants per arm | rho = 0.0 | 0.3 | 0.5")
    for d in (0.05, 0.06, 0.08):
        ns = [per_arm(d, sigma, r) for r in (0.0, 0.3, 0.5)]
        print(f"   CE difference {d:.2f} (d = {d / sigma:.2f})  |"
              f" {ns[0]:9d} | {ns[1]:3d} | {ns[2]:3d}")
    n = per_arm(0.06, sigma, 0.5)
    rec = math.ceil(n / 0.90)
    print(f"chosen: 0.06, rho = 0.5 -> {n} per arm")
    print(f"   allowing 10 % loss (incomplete sessions, failed attention checks):")
    print(f"   recruit {rec} per arm, {2 * rec} in all")
    n0, reps, hits = per_arm(0.06, sigma, 0.0), 1000, 0
    for _ in range(reps):
        a, b = ce_sample(n0, 0.50), ce_sample(n0, 0.57)
        se = math.sqrt(stdev(a) ** 2 / n0 + stdev(b) ** 2 / n0)
        hits += abs(mean(a) - mean(b)) / se > N.inv_cdf(0.975)
    print(f"check without covariate: {n0} per arm, simulated power {hits / reps:.2f}")
    pk = [math.comb(20, k) * 0.8 ** k * 0.2 ** (20 - k) for k in range(21)]
    floor = sum(p * abs(k / 20 - 0.8) for k, p in enumerate(pk))
    print(f"floor: a perfectly calibrated person (h = 0.80) has expected CE {floor:.3f}"
          " on 20 items")

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《概率判断、布莱尔分数校准与群体智慧的统计理论》第 8 章「实践与预注册试验」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23764998

DOI: 10.5281/zenodo.23764998 · 阅读本章所在专著 · 5DL-B-026 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026026,
  author    = {Fan, Yafu},
  title     = {Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.23764998},
  url       = {https://doi.org/10.5281/zenodo.23764998}
}

本书其他章节结论