关于「杰弗里斯先验:不变性及其代价」,《信息几何与自然梯度》给出的定量结论是什么?

结论:杰弗里斯先验 \(\sqrt{\det G}\) 是费希尔—拉奥体积元,因而在任何坐标下应用都得到同一个分布(程序中两种坐标的区间概率都是 0.164177)。但它不是「无信息、总是合适」的:正态均值与泊松率的规则先验不可积;多参数的联合形式在内曼—斯科特问题中使后验以越来越小的标准差集中在真值的一半(0.4934),杰弗里斯本人建议的独立形式没有这个问题(0.9870);它依赖抽样设计,二项与负二项给出不同的后验,违反似然原理。另一方面,它的区间在二项比例上有接近名义水平的频率覆盖。参考先验是更系统的推广。

数据口径:第八章给出四步核查表,以及一项预注册的计算基准试验:由独立第三方冻结一组分层的合成任务,比较自然梯度(精确或 K-FAC)与 Adam 达到目标损失的墙钟时间,唯一主要结局为二者时间比的以 2 为底的对数在任务上的均值,最小关注效应 log₂1.25,90% 把握度需 393 个配对任务、计入无效任务后计划 470 个,判定规则写成代码。 书中以标注框区分已发表的证据、可运行的 Python 计算(16 段程序)、原典与作者的模型和假说;书中数据均为合成或模拟,不是对任何具体机器学习系统性能的保证。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.4934, 0.9870

(1) Neyman-Scott, true sigma^21; posterior mean (sd) of sigma^2
k joint rule sigma^-(k+1) independent 1/sigma
100.3435 (0.1214) 0.7729 (0.4462)
1000.5415 (0.0547) 1.0941 (0.1579)
10,0000.4934 (0.0049) 0.9870 (0.0140)
(2) 3 successes in 12 trials; likelihoods are proportional to p^3 (1-p)^9
binomial Jeffreys Beta(1/2,1/2) -> Beta(3.5, 9.5): mean 0.2692, P(p < 0.1) 0.0555
neg. binomial p^-1 (1-p)^-1/2 -> Beta(3, 9.5): mean 0.2400, P(p < 0.1) 0.1000
(3) exact coverage of nominal 95 % intervals, p on a grid of 99 points
n Wald: mean min Jeffreys: mean min points with coverage < 0.93
100.776 0.096 0.952 0.869 Wald 93/99, Jeffreys 22/99
300.883 0.260 0.950 0.889 Wald 52/99, Jeffreys 8/99
1000.927 0.633 0.950 0.921 Wald 20/99, Jeffreys 2/99

核心公式

\[\pi(\theta) \propto \sqrt{\det G(\theta)} .\]
查看完整验算程序
# Where Jeffreys's rule helps and where it fails. (1) Neyman-Scott: k groups of 2 normal
# observations, separate means, common sigma^2 = 1. The joint rule sqrt(det G) ~
# sigma^-(k+1) gives a posterior for sigma^2 centred near 1/2 however large k is; the
# "independent" prior 1/sigma (Jeffreys's own advice for location-scale) gives ~1.
# (2) Same likelihood, different design: binomial (n fixed) and negative binomial (stop at
# the r-th success) Jeffreys priors differ, so the posteriors differ (likelihood
# principle). (3) Exact frequentist coverage of 95 % intervals for a binomial proportion:
# Wald versus the Jeffreys interval (Beta(x + 1/2, n - x + 1/2) quantiles).
import math, random, statistics

def betacf(a, b, x):                                  # Lentz continued fraction
    tiny = 1e-300; c, d = 1.0, 1 - (a + b) * x / (a + 1)
    d = 1 / (d if abs(d) > tiny else tiny); h = d
    for m in range(1, 300):
        for num in (m * (b - m) * x / ((a + 2 * m - 1) * (a + 2 * m)),
                    -(a + m) * (a + b + m) * x / ((a + 2 * m) * (a + 2 * m + 1))):
            d = 1 + num * d; d = 1 / (d if abs(d) > tiny else tiny)
            c = 1 + num / c; c = c if abs(c) > tiny else tiny; h *= d * c
        if abs(d * c - 1) < 1e-14:
            break
    return h

def ibeta(a, b, x):                                   # regularised incomplete beta
    if x <= 0 or x >= 1:
        return 0.0 if x <= 0 else 1.0
    lb = math.lgamma(a + b) - math.lgamma(a) - math.lgamma(b)
    f = math.exp(lb + a * math.log(x) + b * math.log(1 - x))
    if x < (a + 1) / (a + b + 2):
        return f * betacf(a, b, x) / a
    return 1 - f * betacf(b, a, 1 - x) / b

def bquant(a, b, q):
    lo, hi = 0.0, 1.0
    for _ in range(60):
        mid = (lo + hi) / 2
        lo, hi = (mid, hi) if ibeta(a, b, mid) < q else (lo, mid)
    return (lo + hi) / 2

rng = random.Random(1948)
print("(1) Neyman-Scott, true sigma^2 = 1; posterior mean (sd) of sigma^2")
print("        k   joint rule sigma^-(k+1)   independent 1/sigma")
for k in (10, 100, 10_000):
    S = sum(rng.gauss(0, 1) ** 2 for _ in range(k))     # within-pair SS ~ chi2_k
    for shape in (k, k / 2):                            # inverse-gamma(shape, S/2)
        m = (S / 2) / (shape - 1); sd = m / math.sqrt(shape - 2)
        if shape == k:
            out = f"{m:8.4f} ({sd:.4f})"
        else:
            out += f"          {m:8.4f} ({sd:.4f})"
    print(f"  {k:7,d}   {out}")
x, n, r = 3, 12, 3
mb, mn = (x + 0.5) / (n + 1), x / (n + 0.5)
print(f"(2) {x} successes in {n} trials; likelihoods are proportional to p^3 (1-p)^9")
print(f"   binomial Jeffreys Beta(1/2,1/2) -> Beta(3.5, 9.5): mean {mb:.4f},"
      f" P(p < 0.1) {ibeta(3.5, 9.5, 0.1):.4f}")
print(f"   neg. binomial p^-1 (1-p)^-1/2   -> Beta(3, 9.5):   mean {mn:.4f},"
      f" P(p < 0.1) {ibeta(3, 9.5, 0.1):.4f}")
print("(3) exact coverage of nominal 95 % intervals, p on a grid of 99 points")
z = statistics.NormalDist().inv_cdf(0.975)
print("     n   Wald: mean  min      Jeffreys: mean  min   points with coverage < 0.93")
for n in (10, 30, 100):
    pmf = lambda p: [math.comb(n, k) * p ** k * (1 - p) ** (n - k) for k in range(n + 1)]
    wald, jef = [], []
    for k in range(n + 1):
        ph = k / n; h = z * math.sqrt(ph * (1 - ph) / n)
        wald.append((ph - h, ph + h))
        jl = 0.0 if k == 0 else bquant(k + 0.5, n - k + 0.5, 0.025)
        ju = 1.0 if k == n else bquant(k + 0.5, n - k + 0.5, 0.975)
        jef.append((jl, ju))
    cw, cj = [], []
    for i in range(1, 100):
        p = i / 100; f = pmf(p)
        cw.append(sum(f[k] for k in range(n + 1) if wald[k][0] <= p <= wald[k][1]))
        cj.append(sum(f[k] for k in range(n + 1) if jef[k][0] <= p <= jef[k][1]))
    print(f"   {n:3d}   {statistics.fmean(cw):.3f}  {min(cw):.3f}       "
          f"{statistics.fmean(cj):.3f}  {min(cj):.3f}   Wald {sum(c < .93 for c in cw)}"
          f"/99, Jeffreys {sum(c < .93 for c in cj)}/99")

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《信息几何、费希尔信息度量与自然梯度优化的统计流形》第 3 章「杰弗里斯先验:不变性及其代价」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23836268

DOI: 10.5281/zenodo.23836268 · 阅读本章所在专著 · 5DL-B-049 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026049,
  author    = {Fan, Yafu},
  title     = {Information Geometry, the Fisher Information Metric and Natural-Gradient Optimisation on Statistical Manifolds},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.23836268},
  url       = {https://doi.org/10.5281/zenodo.23836268}
}

本书其他章节结论