关于「实践与预注册试验」,《概率校准与群体智慧》给出的定量结论是什么?
结论:四步校准核查表把前几章的结论落到四个可记录的环节:用数字替代模糊词,先找外部参考类别的基础率,事前写下更新触发条件,季度结束后用布莱尔分数复盘并与基础率比较;虚构示例显示,直觉数字可能比只报基础率还差,而按事先约定更新后有所改善,但一个季度的 12 个问题不足以下结论。试验方案比较 15 分钟校准微训练与时长匹配的逻辑思维阅读,唯一的主要结局是 20 道题上 80% 区间的校准误差 \(|H - 0.80|\),每组 133 人可分析、招募 148 人,判定规则包括操作检查与排除比例的门槛,写成代码并事先公开;无论结果正负都发表。
数据口径:本书不是心理咨询、医学或投资建议。 书中以标注框区分已发表的证据、可运行的 Python 计算(16 段程序,全部核对)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.80, 133, 148, 20
| simulated control: mean CE 0.304, SD 0.173; hit chance 0.57 gives mean CE 0.243 | |
| analysable participants per arm | rho | 0.0 | 0.3 | 0.5 |
| CE difference 0.05 (d | 0.29) | 253 | 231 | 190 |
| CE difference 0.06 (d | 0.35) | 176 | 161 | 133 |
| CE difference 0.08 (d | 0.46) | 100 | 91 | 75 |
| chosen | 0.06, rho = 0.5 -> 133 per arm |
| allowing 10 % loss (incomplete sessions, failed attention checks): | |
| recruit 148 per arm, 296 in all | |
| check without covariate | 176 per arm, simulated power 0.93 |
| floor: a perfectly calibrated person (h | 0.80) has expected CE 0.070 on 20 items |
查看完整验算程序
# Sample size. Primary outcome CE = |H - 0.80| over 20 items (H = share of 80 %
# intervals that hit). ANCOVA on pre-test CE (correlation rho); alpha 0.05 two-sided,
# power 0.90. SD of CE from a simulated control arm: hit chance ~ Beta(mean 0.50, SD
# 0.15), an assumption.
import math, random
from statistics import NormalDist, mean, stdev
rng = random.Random(15)
N = NormalDist()
def beta_ab(m, sd):
k = m * (1 - m) / sd ** 2 - 1
return m * k, (1 - m) * k
def ce_sample(n, m, sd=0.15, items=20):
a, b = beta_ab(m, sd)
out = []
for _ in range(n):
h = rng.betavariate(a, b)
hits = sum(rng.random() < h for _ in range(items))
out.append(abs(hits / items - 0.80))
return out
def per_arm(delta, sigma, rho, alpha=0.05, power=0.90):
z = N.inv_cdf(1 - alpha / 2) + N.inv_cdf(power)
return math.ceil(2 * z * z * sigma ** 2 * (1 - rho ** 2) / delta ** 2) + 1
if __name__ == "__main__":
ctrl, trt = ce_sample(20000, 0.50), ce_sample(20000, 0.57)
sigma = stdev(ctrl)
print(f"simulated control: mean CE {mean(ctrl):.3f}, SD {sigma:.3f};"
f" hit chance 0.57 gives mean CE {mean(trt):.3f}")
print("analysable participants per arm | rho = 0.0 | 0.3 | 0.5")
for d in (0.05, 0.06, 0.08):
ns = [per_arm(d, sigma, r) for r in (0.0, 0.3, 0.5)]
print(f" CE difference {d:.2f} (d = {d / sigma:.2f}) |"
f" {ns[0]:9d} | {ns[1]:3d} | {ns[2]:3d}")
n = per_arm(0.06, sigma, 0.5)
rec = math.ceil(n / 0.90)
print(f"chosen: 0.06, rho = 0.5 -> {n} per arm")
print(f" allowing 10 % loss (incomplete sessions, failed attention checks):")
print(f" recruit {rec} per arm, {2 * rec} in all")
n0, reps, hits = per_arm(0.06, sigma, 0.0), 1000, 0
for _ in range(reps):
a, b = ce_sample(n0, 0.50), ce_sample(n0, 0.57)
se = math.sqrt(stdev(a) ** 2 / n0 + stdev(b) ** 2 / n0)
hits += abs(mean(a) - mean(b)) / se > N.inv_cdf(0.975)
print(f"check without covariate: {n0} per arm, simulated power {hits / reps:.2f}")
pk = [math.comb(20, k) * 0.8 ** k * 0.2 ** (20 - k) for k in range(21)]
floor = sum(p * abs(k / 20 - 0.8) for k, p in enumerate(pk))
print(f"floor: a perfectly calibrated person (h = 0.80) has expected CE {floor:.3f}"
" on 20 items")唯一数据源与引用
范雅福(Fan, Y.). (2026). 《概率判断、布莱尔分数校准与群体智慧的统计理论》第 8 章「实践与预注册试验」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23764998
DOI: 10.5281/zenodo.23764998 · 阅读本章所在专著 · 5DL-B-026 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026026,
author = {Fan, Yafu},
title = {Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.23764998},
url = {https://doi.org/10.5281/zenodo.23764998}
}