关于「杰弗里斯先验:不变性及其代价」,《信息几何与自然梯度》给出的定量结论是什么?
结论:杰弗里斯先验 \(\sqrt{\det G}\) 是费希尔—拉奥体积元,因而在任何坐标下应用都得到同一个分布(程序中两种坐标的区间概率都是 0.164177)。但它不是「无信息、总是合适」的:正态均值与泊松率的规则先验不可积;多参数的联合形式在内曼—斯科特问题中使后验以越来越小的标准差集中在真值的一半(0.4934),杰弗里斯本人建议的独立形式没有这个问题(0.9870);它依赖抽样设计,二项与负二项给出不同的后验,违反似然原理。另一方面,它的区间在二项比例上有接近名义水平的频率覆盖。参考先验是更系统的推广。
数据口径:第八章给出四步核查表,以及一项预注册的计算基准试验:由独立第三方冻结一组分层的合成任务,比较自然梯度(精确或 K-FAC)与 Adam 达到目标损失的墙钟时间,唯一主要结局为二者时间比的以 2 为底的对数在任务上的均值,最小关注效应 log₂1.25,90% 把握度需 393 个配对任务、计入无效任务后计划 470 个,判定规则写成代码。 书中以标注框区分已发表的证据、可运行的 Python 计算(16 段程序)、原典与作者的模型和假说;书中数据均为合成或模拟,不是对任何具体机器学习系统性能的保证。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.4934, 0.9870
| (1) Neyman-Scott, true sigma^2 | 1; posterior mean (sd) of sigma^2 |
| k joint rule sigma^-(k+1) independent 1/sigma | |
| 10 | 0.3435 (0.1214) 0.7729 (0.4462) |
| 100 | 0.5415 (0.0547) 1.0941 (0.1579) |
| 10,000 | 0.4934 (0.0049) 0.9870 (0.0140) |
| (2) 3 successes in 12 trials; likelihoods are proportional to p^3 (1-p)^9 | |
| binomial Jeffreys Beta(1/2,1/2) -> Beta(3.5, 9.5): mean 0.2692, P(p < 0.1) 0.0555 | |
| neg. binomial p^-1 (1-p)^-1/2 -> Beta(3, 9.5): mean 0.2400, P(p < 0.1) 0.1000 | |
| (3) exact coverage of nominal 95 % intervals, p on a grid of 99 points | |
| n Wald: mean min Jeffreys: mean min points with coverage < 0.93 | |
| 10 | 0.776 0.096 0.952 0.869 Wald 93/99, Jeffreys 22/99 |
| 30 | 0.883 0.260 0.950 0.889 Wald 52/99, Jeffreys 8/99 |
| 100 | 0.927 0.633 0.950 0.921 Wald 20/99, Jeffreys 2/99 |
核心公式
查看完整验算程序
# Where Jeffreys's rule helps and where it fails. (1) Neyman-Scott: k groups of 2 normal
# observations, separate means, common sigma^2 = 1. The joint rule sqrt(det G) ~
# sigma^-(k+1) gives a posterior for sigma^2 centred near 1/2 however large k is; the
# "independent" prior 1/sigma (Jeffreys's own advice for location-scale) gives ~1.
# (2) Same likelihood, different design: binomial (n fixed) and negative binomial (stop at
# the r-th success) Jeffreys priors differ, so the posteriors differ (likelihood
# principle). (3) Exact frequentist coverage of 95 % intervals for a binomial proportion:
# Wald versus the Jeffreys interval (Beta(x + 1/2, n - x + 1/2) quantiles).
import math, random, statistics
def betacf(a, b, x): # Lentz continued fraction
tiny = 1e-300; c, d = 1.0, 1 - (a + b) * x / (a + 1)
d = 1 / (d if abs(d) > tiny else tiny); h = d
for m in range(1, 300):
for num in (m * (b - m) * x / ((a + 2 * m - 1) * (a + 2 * m)),
-(a + m) * (a + b + m) * x / ((a + 2 * m) * (a + 2 * m + 1))):
d = 1 + num * d; d = 1 / (d if abs(d) > tiny else tiny)
c = 1 + num / c; c = c if abs(c) > tiny else tiny; h *= d * c
if abs(d * c - 1) < 1e-14:
break
return h
def ibeta(a, b, x): # regularised incomplete beta
if x <= 0 or x >= 1:
return 0.0 if x <= 0 else 1.0
lb = math.lgamma(a + b) - math.lgamma(a) - math.lgamma(b)
f = math.exp(lb + a * math.log(x) + b * math.log(1 - x))
if x < (a + 1) / (a + b + 2):
return f * betacf(a, b, x) / a
return 1 - f * betacf(b, a, 1 - x) / b
def bquant(a, b, q):
lo, hi = 0.0, 1.0
for _ in range(60):
mid = (lo + hi) / 2
lo, hi = (mid, hi) if ibeta(a, b, mid) < q else (lo, mid)
return (lo + hi) / 2
rng = random.Random(1948)
print("(1) Neyman-Scott, true sigma^2 = 1; posterior mean (sd) of sigma^2")
print(" k joint rule sigma^-(k+1) independent 1/sigma")
for k in (10, 100, 10_000):
S = sum(rng.gauss(0, 1) ** 2 for _ in range(k)) # within-pair SS ~ chi2_k
for shape in (k, k / 2): # inverse-gamma(shape, S/2)
m = (S / 2) / (shape - 1); sd = m / math.sqrt(shape - 2)
if shape == k:
out = f"{m:8.4f} ({sd:.4f})"
else:
out += f" {m:8.4f} ({sd:.4f})"
print(f" {k:7,d} {out}")
x, n, r = 3, 12, 3
mb, mn = (x + 0.5) / (n + 1), x / (n + 0.5)
print(f"(2) {x} successes in {n} trials; likelihoods are proportional to p^3 (1-p)^9")
print(f" binomial Jeffreys Beta(1/2,1/2) -> Beta(3.5, 9.5): mean {mb:.4f},"
f" P(p < 0.1) {ibeta(3.5, 9.5, 0.1):.4f}")
print(f" neg. binomial p^-1 (1-p)^-1/2 -> Beta(3, 9.5): mean {mn:.4f},"
f" P(p < 0.1) {ibeta(3, 9.5, 0.1):.4f}")
print("(3) exact coverage of nominal 95 % intervals, p on a grid of 99 points")
z = statistics.NormalDist().inv_cdf(0.975)
print(" n Wald: mean min Jeffreys: mean min points with coverage < 0.93")
for n in (10, 30, 100):
pmf = lambda p: [math.comb(n, k) * p ** k * (1 - p) ** (n - k) for k in range(n + 1)]
wald, jef = [], []
for k in range(n + 1):
ph = k / n; h = z * math.sqrt(ph * (1 - ph) / n)
wald.append((ph - h, ph + h))
jl = 0.0 if k == 0 else bquant(k + 0.5, n - k + 0.5, 0.025)
ju = 1.0 if k == n else bquant(k + 0.5, n - k + 0.5, 0.975)
jef.append((jl, ju))
cw, cj = [], []
for i in range(1, 100):
p = i / 100; f = pmf(p)
cw.append(sum(f[k] for k in range(n + 1) if wald[k][0] <= p <= wald[k][1]))
cj.append(sum(f[k] for k in range(n + 1) if jef[k][0] <= p <= jef[k][1]))
print(f" {n:3d} {statistics.fmean(cw):.3f} {min(cw):.3f} "
f"{statistics.fmean(cj):.3f} {min(cj):.3f} Wald {sum(c < .93 for c in cw)}"
f"/99, Jeffreys {sum(c < .93 for c in cj)}/99")唯一数据源与引用
范雅福(Fan, Y.). (2026). 《信息几何、费希尔信息度量与自然梯度优化的统计流形》第 3 章「杰弗里斯先验:不变性及其代价」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23836268
DOI: 10.5281/zenodo.23836268 · 阅读本章所在专著 · 5DL-B-049 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026049,
author = {Fan, Yafu},
title = {Information Geometry, the Fisher Information Metric and Natural-Gradient Optimisation on Statistical Manifolds},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.23836268},
url = {https://doi.org/10.5281/zenodo.23836268}
}