关于「实践与预注册检验」,《信息几何与自然梯度》给出的定量结论是什么?

结论:第八章给出四步核查表,以及一项预注册的计算基准试验:由独立第三方冻结一组分层的合成任务,比较自然梯度(精确或 K-FAC)与 Adam 达到目标损失的墙钟时间,唯一主要结局为二者时间比的以 2 为底的对数在任务上的均值,最小关注效应 log₂1.25,90% 把握度需 393 个配对任务、计入无效任务后计划 470 个,判定规则写成代码。书中以标注框区分已发表的证据、可运行的 Python 计算(16 段程序)、原典与作者的模型和假说;书中数据均为合成或模拟,不是对任何具体机器学习系统性能的保证。

数据口径:第八章给出四步核查表,以及一项预注册的计算基准试验:由独立第三方冻结一组分层的合成任务,比较自然梯度(精确或 K-FAC)与 Adam 达到目标损失的墙钟时间,唯一主要结局为二者时间比的以 2 为底的对数在任务上的均值,最小关注效应 log₂1.25,90% 把握度需 393 个配对任务、计入无效任务后计划 470 个,判定规则写成代码。 书中以标注框区分已发表的证据、可运行的 Python 计算(16 段程序)、原典与作者的模型和假说;书中数据均为合成或模拟,不是对任何具体机器学习系统性能的保证。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 1.25, 393, 470, 90

pilot (40 tasks): mean D -2.78 (NG faster if < 0), sd 1.96,
   range -6.61 to +0.36; Adam missed the target on 4 task(s) (time set to 2 x cap)
SESOI log2(1.25) = 0.322; 90 % power, two-sided alpha 0.05: n = 392.5 -> 393 tasks
   simulation (2,000 resampled suites of 393, true mean D = -SESOI): power 0.906
   allow 15 % lost to infrastructure faults: generate 470 (94 in each of 5 strata)
sensitivity: tasks needed (before the 15 % allowance)
   sd of D:     0.98   1.96   2.94   3.93
   n:            100    393    881   1565
blinded re-estimation after 25 % of tasks: sd of D -> tasks to generate
   1.5 -> 470  2.0 -> 480  2.5 -> 750  3.0 -> 1,080  4.0 -> 1,920
查看完整验算程序
# Sample size for the Chapter 8 pre-registered benchmark. Unit = a synthetic task drawn
# by the third party; both optimisers run on it (paired). Primary outcome per task:
# D = log2(time to target, natural gradient / time to target, Adam). Estimand: mean D
# over the task population (log2 of the geometric-mean time ratio). SESOI log2(1.25).
# PILOT (planning assumption only, not the trial suite): 40 logistic-regression tasks,
# n = 40, d = 4, kappa log-uniform on [1, 10^4], random rotation; "time" = gradient-
# evaluation units (Newton/NG step = 1 + d + d^2/(3n)); Adam: best of 3 steps, cap 500.
# Last part: the registered blinded re-estimation rule (sd of D from the first 25 % of
# tasks; sd is unchanged by swapping the arm labels; n may only go up, cap 2,000).
import math, random, statistics

NZ = statistics.NormalDist(); Z = NZ.inv_cdf

def task(rng, n=40, d=4):
    kappa = 10 ** rng.uniform(0, 4); ev = [kappa ** (-i / (d - 1)) for i in range(d)]
    Q = []
    while len(Q) < d:
        v = [rng.gauss(0, 1) for _ in range(d)]
        for q in Q:
            c = sum(a * b for a, b in zip(v, q)); v = [a - c * b for a, b in zip(v, q)]
        s = math.sqrt(sum(a * a for a in v)); Q.append([a / s for a in v])
    X = []
    for _ in range(n):
        z = [rng.gauss(0, 1) * math.sqrt(e) for e in ev]
        X.append([sum(Q[i][j] * z[j] for j in range(d)) for i in range(d)])
    w = [sum(Q[i][j] / math.sqrt(ev[j]) for j in range(d)) / d for i in range(d)]
    y = [int(rng.random() < 1 / (1 + math.exp(-sum(a * b for a, b in zip(w, x)))))
         for x in X]
    return X, y

def lgf(X, y, w, hess=False):
    d, n = len(w), len(X); L, g = 0.0, [0.0] * d; H = [[0.0] * d for _ in range(d)]
    for x, t in zip(X, y):
        z = sum(a * b for a, b in zip(w, x))
        p = 1 / (1 + math.exp(-z)) if z >= 0 else math.exp(z) / (1 + math.exp(z))
        L += math.log1p(math.exp(-abs(z))) + max(z, 0) - t * z
        for i in range(d):
            g[i] += (p - t) * x[i] / n
            if hess:
                for j in range(d):
                    H[i][j] += p * (1 - p) * x[i] * x[j] / n
    return L / n, g, H

def newton(X, y, w):
    L, g, H = lgf(X, y, w, True); d = len(w)
    M = [H[i][:] + [g[i]] for i in range(d)]
    for i in range(d):
        M[i][i] += 1e-8
    for c in range(d):
        for r in range(c + 1, d):
            f = M[r][c] / M[c][c]; M[r] = [a - f * b for a, b in zip(M[r], M[c])]
    s = [0.0] * d
    for r in range(d - 1, -1, -1):
        s[r] = (M[r][d] - sum(M[r][k] * s[k] for k in range(r + 1, d))) / M[r][r]
    return [a - b for a, b in zip(w, s)], L

def pilot_D(seed):
    rng = random.Random(seed); X, y = task(rng); d, n = len(X[0]), len(X)
    w = [0.0] * d
    for _ in range(30):
        w, _ = newton(X, y, w)
    target = lgf(X, y, w)[0] + 1e-3; w = [0.0] * d; k_ng = 0
    while lgf(X, y, w)[0] > target:
        w, _ = newton(X, y, w); k_ng += 1
    best = None
    for lr in (0.03, 0.1, 0.3):
        w = [0.0] * d; m = [0.0] * d; v = [0.0] * d
        for k in range(1, 501):
            L, g, _ = lgf(X, y, w)
            if L <= target:
                best = k - 1 if best is None else min(best, k - 1); break
            m = [0.9 * a + 0.1 * b for a, b in zip(m, g)]
            v = [0.999 * a + 0.001 * b * b for a, b in zip(v, g)]
            w = [a - lr * (b / (1 - 0.9 ** k)) / (math.sqrt(c / (1 - 0.999 ** k)) + 1e-8)
                 for a, b, c in zip(w, m, v)]
    t_ng = k_ng * (1 + d + d * d / (3 * n)); t_ad = best if best is not None else 1000
    return math.log2(max(t_ng, 1) / max(t_ad, 1)), best is None

NP = 40
Ds = [pilot_D(s) for s in range(NP)]
D = [a for a, _ in Ds]; sd = statistics.stdev(D)
print(f"pilot ({NP} tasks): mean D {statistics.fmean(D):+.2f} (NG faster if < 0),"
      f" sd {sd:.2f},"
      f"\n   range {min(D):+.2f} to {max(D):+.2f}; Adam missed the target on"
      f" {sum(f for _, f in Ds)} task(s) (time set to 2 x cap)")
delta = math.log2(1.25); za, zb = Z(0.975), Z(0.90)
n0 = ((za + zb) * sd / delta) ** 2 + za * za / 2
n_t = math.ceil(n0)
print(f"SESOI log2(1.25) = {delta:.3f}; 90 % power, two-sided alpha 0.05:"
      f" n = {n0:.1f} -> {n_t} tasks")
rng = random.Random(15); hits = 0; R = 2000
cen = [a - statistics.fmean(D) - delta for a in D]       # true mean D = -SESOI
for _ in range(R):
    s = [rng.choice(cen) for _ in range(n_t)]
    m, se = statistics.fmean(s), statistics.stdev(s) / math.sqrt(n_t)
    hits += m + 1.96 * se < 0                              # "NG faster" shown
print(f"   simulation ({R:,} resampled suites of {n_t}, true mean D = -SESOI):"
      f" power {hits / R:.3f}")
plan = math.ceil(n_t / 0.85 / 10) * 10
print(f"   allow 15 % lost to infrastructure faults: generate {plan}"
      f" ({plan // 5} in each of 5 strata)")
print("sensitivity: tasks needed (before the 15 % allowance)")
SDS = (0.5 * sd, sd, 1.5 * sd, 2 * sd)
print("   sd of D:    " + "  ".join(f"{s:5.2f}" for s in SDS))
need = lambda s: math.ceil(((za + zb) * s / delta) ** 2 + za * za / 2)
print("   n:          " + "  ".join(f"{need(s):5d}" for s in SDS))

def reestimate(sd_blind, n_plan=plan, cap=2000):     # registered rule, never decreases
    n_new = math.ceil(need(sd_blind) / 0.85 / 10) * 10
    return min(cap, max(n_plan, n_new))
print("blinded re-estimation after 25 % of tasks: sd of D -> tasks to generate")
SD2 = (1.5, 2.0, 2.5, 3.0, 4.0)
print("   " + "  ".join(f"{s:.1f} -> {reestimate(s):,}" for s in SD2))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《信息几何、费希尔信息度量与自然梯度优化的统计流形》第 8 章「实践与预注册检验」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23836268

DOI: 10.5281/zenodo.23836268 · 阅读本章所在专著 · 5DL-B-049 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026049,
  author    = {Fan, Yafu},
  title     = {Information Geometry, the Fisher Information Metric and Natural-Gradient Optimisation on Statistical Manifolds},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.23836268},
  url       = {https://doi.org/10.5281/zenodo.23836268}
}

本书其他章节结论