5D Library › 百卷知识超图拓扑、多模态个人先验编译与自主知识帝国标度律 · English

第二章 另外三个原语:威布尔风险率、勒让德-布雷格曼对偶与贝叶斯更新

一条风险率定律

本书库的许多卷都在问某样东西能持续多久:观众停留在一段视频上、订阅者、存储的文件。威布尔定律用两个数描述这三者:尺度与形状;形状参数说明风险率是下降(小于 1)、不变(等于 1)还是上升(大于 1)。p03 用同一个最大似然程序拟合三个领域的合成数据。

推导

# Chapter 2. p03: One hazard law, three domains. Weibull survival
# S(t) = exp(-(t / lam)^k) has hazard (k / lam) (t / lam)^(k - 1):
# falling if k < 1, flat if k = 1, rising if k > 1. Synthetic data from
# three illustrative domains, 2,000 cases each: seconds a viewer stays
# with a short video (k 0.6, lam 20); months a subscriber stays (k 0.9,
# lam 14); years until a storage medium fails (k 1.8, lam 12). The same
# maximum-likelihood routine (grid over k, lam in closed form) fits
# each. Seed 100.
import random
from math import log

def fit(ts):
    n, slog = len(ts), sum(log(t) for t in ts)
    best = None
    for i in range(1, 400):
        k = i / 100
        lam = (sum(t ** k for t in ts) / n) ** (1 / k)
        ll = n * log(k) - n * k * log(lam) + (k - 1) * slog - n
        if best is None or ll > best[0]:
            best = (ll, k, lam)
    return best[1], best[2]

DOMAINS = (("video watch", "s", 0.6, 20.0), ("subscription", "mo", 0.9,
           14.0), ("storage medium", "yr", 1.8, 12.0))

if __name__ == "__main__":
    rng = random.Random(100)
    print("   domain          true k  fitted k  fitted lam  median  "
          "hazard t=1 / t=10")
    for name, unit, k, lam in DOMAINS:
        ts = [rng.weibullvariate(lam, k) for _ in range(2000)]
        fk, fl = fit(ts)
        med = fl * log(2) ** (1 / fk)
        h = lambda t: (fk / fl) * (t / fl) ** (fk - 1)
        fmt = ("   {:<14}  {:6.2f}  {:8.2f}  {:7.1f} {:<2}  {:6.1f}"
               "  {:.3f} / {:.3f}")
        print(fmt.format(name, k, fk, fl, unit, med, h(1), h(10)))
#    domain          true k  fitted k  fitted lam  median  hazard t=1 / t=10
#    video watch       0.60      0.60     21.1 s     11.5  0.096 / 0.038
#    subscription      0.90      0.91     13.8 mo     9.2  0.084 / 0.068
#    storage medium    1.80      1.81     12.0 yr     9.8  0.020 / 0.131

拟合各从 2,000 个案例中还原了形状参数(0.60、0.91 与 1.81,真实值为 0.60、0.90 与 1.80)。视频观众的风险率从第一秒的 0.096 降到第十秒的 0.038;订阅者略有下降;存储介质则从每年 0.020 升到 0.131。「留存」与「比特腐败」是形状相反的同一个计算。

一个凸函数

预测市场的对数市场评分规则 (Hanson 2003) 与神经网络注意力层的 softmax,是同一个凸函数,即对数求和指数的梯度;它的凸共轭是负熵,而负熵的布雷格曼散度就是 KL 散度 (Banerjee et al. 2005)。p04 以数值核对这一点。

推导

# Chapter 2. p04: One convex function, two machines. The log-sum-exp
# C(q) = b log(sum exp(q_i / b)) is the cost function of Hanson's
# logarithmic market scoring rule; its gradient, softmax(q / b), is both
# the market's price vector and the weight vector of an attention layer
# or a logit model. Its convex conjugate is b times negative entropy,
# C*(p) = b sum p_i log p_i, and the Bregman divergence of C* is b times
# the Kullback-Leibler divergence. Checked numerically for q = (2, 1,
# 0, -1), b = 1.5, and a trader who believes pi = (0.5, 0.3, 0.15,
# 0.05) and trades until the prices equal pi.
from math import exp, log

B = 1.5
Q = (2.0, 1.0, 0.0, -1.0)
PI = (0.5, 0.3, 0.15, 0.05)

def cost(q):
    return B * log(sum(exp(x / B) for x in q))

def softmax(q):
    z = [exp(x / B) for x in q]
    return [v / sum(z) for v in z]

def neg_entropy(p):
    return B * sum(x * log(x) for x in p if x > 0)

def kl(a, b):
    return sum(x * log(x / y) for x, y in zip(a, b) if x > 0)

if __name__ == "__main__":
    p = softmax(Q)
    print("   prices = attention weights: " + "  ".join(
        "{:.4f}".format(x) for x in p))
    gap = cost(Q) + neg_entropy(p) - sum(a * b for a, b in zip(Q, p))
    print("   Fenchel-Young gap at the gradient below 1e-12: {}".format(
        "yes" if abs(gap) < 1e-12 else "no"))
    q2 = [B * log(x) + cost(Q) for x in PI]
    paid = cost(q2) - cost(Q)
    gain = sum(pi * (a - b) for pi, a, b in zip(PI, q2, Q))
    print("   trader moves prices to pi: net payment {:.4f}, expected"
          " payout {:.4f}".format(abs(paid), gain))
    print("   expected profit {:.4f} = b x KL(pi || p) = {:.4f}".format(
        gain - paid, B * kl(PI, p)))
#    prices = attention weights: 0.5229  0.2685  0.1378  0.0708
#    Fenchel-Young gap at the gradient below 1e-12: yes
#    trader moves prices to pi: net payment 0.0000, expected payout 0.0093
#    expected profit 0.0093 = b x KL(pi || p) = 0.0093

四个价格(0.5229、0.2685、0.1378、0.0708)就是同一组分数的注意力权重;梯度处的 Fenchel-Young 间隙在机器精度内为零;一位把价格推到自己信念的交易者,预期收益是 0.0093,恰好等于流动性参数乘以 KL 散度。预测市场为信息支付的,与分类器的损失为错误收取的,是同一个量。

一条更新规则

三者中的最后一个,是最简单的共轭形式的贝叶斯法则。p05 把同一个贝塔-二项更新用于结账页的转化率、诊断站用户随后打开对应专著的比例,以及每年校验和失败的概率。

推导

# Chapter 2. p05: One update rule, three uses. A Beta(a, b) prior on a
# rate, after s successes in n trials, becomes Beta(a + s, b + n - s).
# (i) A checkout's conversion rate: prior Beta(2, 48), 30 sales in 500
# visits. (ii) The share of a diagnostic station's users who later open
# the matching book: prior Beta(1, 9), 12 of 80. (iii) The chance that
# a stored file fails its yearly checksum: prior Beta(1, 999), no
# failures in 2,000 checks. Posterior mean, and 95% interval from
# 200,000 draws. Seed 100.
import random

CASES = (("conversion", 2, 48, 30, 500), ("station to book", 1, 9, 12, 80),
         ("checksum failure", 1, 999, 0, 2000))

if __name__ == "__main__":
    rng = random.Random(100)
    print("   use               prior  data       posterior  95% interval")
    for name, a, b, s, n in CASES:
        a2, b2 = a + s, b + n - s
        xs = sorted(rng.betavariate(a2, b2) for _ in range(200000))
        fmt = "   {:<16}  {:.4f}  {:>4}/{:<5}  {:9.5f}  {:.5f} to {:.5f}"
        print(fmt.format(name, a / (a + b), s, n, a2 / (a2 + b2),
                      xs[5000], xs[195000]))
#    use               prior  data       posterior  95% interval
#    conversion        0.0400    30/500      0.05818  0.04024 to 0.07917
#    station to book   0.1000    12/80       0.14444  0.07974 to 0.22355
#    checksum failure  0.0010     0/2000     0.00033  0.00001 to 0.00123

转化率的估计从先验均值 0.0400 移到 0.05818;诊断站到专著的比例从 0.1000 移到 0.14444,95% 区间很宽(0.07974 至 0.22355),因为只看到了 80 名用户;而 2,000 次干净的校验,给失败率留下每年 0.00123 的 95% 上限。规则相同;它能教给我们多少,只取决于每个领域提供了多少数据。

小结

风险率定律、对数求和指数的对偶与共轭更新,是本书库中反复出现的另外三块数学。每一块都可以原封不动地跨领域运行,而在每一块中,领域提供的是数据与参数,而不是形式。

参考文献

Banerjee, Arindam, Srujana Merugu, Inderjit S. Dhillon, and Joydeep Ghosh. 2005. “Clustering with Bregman Divergences.” Journal of Machine Learning Research 6: 1705 to 1749.
Hanson, Robin. 2003. “Combinatorial Information Market Design.” Information Systems Frontiers 5 (1): 107 to 119.

Fan, Y. (2026). Hundred-Volume Knowledge Hypergraphs, Multimodal Personal Prior Compilation and Autonomous Empire Scaling: Cross-Disciplinary Isomorphisms, Zero-Marginal-Cost Replication and Century-Scale Knowledge Preservation [百卷知识超图拓扑、多模态个人先验编译与自主知识帝国标度律] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090260 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).