5D Library › 百卷知识超图拓扑、多模态个人先验编译与自主知识帝国标度律 · English
第二章 另外三个原语:威布尔风险率、勒让德-布雷格曼对偶与贝叶斯更新
一条风险率定律
本书库的许多卷都在问某样东西能持续多久:观众停留在一段视频上、订阅者、存储的文件。威布尔定律用两个数描述这三者:尺度与形状;形状参数说明风险率是下降(小于 1)、不变(等于 1)还是上升(大于 1)。p03 用同一个最大似然程序拟合三个领域的合成数据。
推导
# Chapter 2. p03: One hazard law, three domains. Weibull survival
# S(t) = exp(-(t / lam)^k) has hazard (k / lam) (t / lam)^(k - 1):
# falling if k < 1, flat if k = 1, rising if k > 1. Synthetic data from
# three illustrative domains, 2,000 cases each: seconds a viewer stays
# with a short video (k 0.6, lam 20); months a subscriber stays (k 0.9,
# lam 14); years until a storage medium fails (k 1.8, lam 12). The same
# maximum-likelihood routine (grid over k, lam in closed form) fits
# each. Seed 100.
import random
from math import log
def fit(ts):
n, slog = len(ts), sum(log(t) for t in ts)
best = None
for i in range(1, 400):
k = i / 100
lam = (sum(t ** k for t in ts) / n) ** (1 / k)
ll = n * log(k) - n * k * log(lam) + (k - 1) * slog - n
if best is None or ll > best[0]:
best = (ll, k, lam)
return best[1], best[2]
DOMAINS = (("video watch", "s", 0.6, 20.0), ("subscription", "mo", 0.9,
14.0), ("storage medium", "yr", 1.8, 12.0))
if __name__ == "__main__":
rng = random.Random(100)
print(" domain true k fitted k fitted lam median "
"hazard t=1 / t=10")
for name, unit, k, lam in DOMAINS:
ts = [rng.weibullvariate(lam, k) for _ in range(2000)]
fk, fl = fit(ts)
med = fl * log(2) ** (1 / fk)
h = lambda t: (fk / fl) * (t / fl) ** (fk - 1)
fmt = (" {:<14} {:6.2f} {:8.2f} {:7.1f} {:<2} {:6.1f}"
" {:.3f} / {:.3f}")
print(fmt.format(name, k, fk, fl, unit, med, h(1), h(10)))
# domain true k fitted k fitted lam median hazard t=1 / t=10
# video watch 0.60 0.60 21.1 s 11.5 0.096 / 0.038
# subscription 0.90 0.91 13.8 mo 9.2 0.084 / 0.068
# storage medium 1.80 1.81 12.0 yr 9.8 0.020 / 0.131拟合各从 2,000 个案例中还原了形状参数(0.60、0.91 与 1.81,真实值为 0.60、0.90 与 1.80)。视频观众的风险率从第一秒的 0.096 降到第十秒的 0.038;订阅者略有下降;存储介质则从每年 0.020 升到 0.131。「留存」与「比特腐败」是形状相反的同一个计算。
一个凸函数
预测市场的对数市场评分规则 (Hanson 2003) 与神经网络注意力层的 softmax,是同一个凸函数,即对数求和指数的梯度;它的凸共轭是负熵,而负熵的布雷格曼散度就是 KL 散度 (Banerjee et al. 2005)。p04 以数值核对这一点。
推导
# Chapter 2. p04: One convex function, two machines. The log-sum-exp
# C(q) = b log(sum exp(q_i / b)) is the cost function of Hanson's
# logarithmic market scoring rule; its gradient, softmax(q / b), is both
# the market's price vector and the weight vector of an attention layer
# or a logit model. Its convex conjugate is b times negative entropy,
# C*(p) = b sum p_i log p_i, and the Bregman divergence of C* is b times
# the Kullback-Leibler divergence. Checked numerically for q = (2, 1,
# 0, -1), b = 1.5, and a trader who believes pi = (0.5, 0.3, 0.15,
# 0.05) and trades until the prices equal pi.
from math import exp, log
B = 1.5
Q = (2.0, 1.0, 0.0, -1.0)
PI = (0.5, 0.3, 0.15, 0.05)
def cost(q):
return B * log(sum(exp(x / B) for x in q))
def softmax(q):
z = [exp(x / B) for x in q]
return [v / sum(z) for v in z]
def neg_entropy(p):
return B * sum(x * log(x) for x in p if x > 0)
def kl(a, b):
return sum(x * log(x / y) for x, y in zip(a, b) if x > 0)
if __name__ == "__main__":
p = softmax(Q)
print(" prices = attention weights: " + " ".join(
"{:.4f}".format(x) for x in p))
gap = cost(Q) + neg_entropy(p) - sum(a * b for a, b in zip(Q, p))
print(" Fenchel-Young gap at the gradient below 1e-12: {}".format(
"yes" if abs(gap) < 1e-12 else "no"))
q2 = [B * log(x) + cost(Q) for x in PI]
paid = cost(q2) - cost(Q)
gain = sum(pi * (a - b) for pi, a, b in zip(PI, q2, Q))
print(" trader moves prices to pi: net payment {:.4f}, expected"
" payout {:.4f}".format(abs(paid), gain))
print(" expected profit {:.4f} = b x KL(pi || p) = {:.4f}".format(
gain - paid, B * kl(PI, p)))
# prices = attention weights: 0.5229 0.2685 0.1378 0.0708
# Fenchel-Young gap at the gradient below 1e-12: yes
# trader moves prices to pi: net payment 0.0000, expected payout 0.0093
# expected profit 0.0093 = b x KL(pi || p) = 0.0093四个价格(0.5229、0.2685、0.1378、0.0708)就是同一组分数的注意力权重;梯度处的 Fenchel-Young 间隙在机器精度内为零;一位把价格推到自己信念的交易者,预期收益是 0.0093,恰好等于流动性参数乘以 KL 散度。预测市场为信息支付的,与分类器的损失为错误收取的,是同一个量。
一条更新规则
三者中的最后一个,是最简单的共轭形式的贝叶斯法则。p05 把同一个贝塔-二项更新用于结账页的转化率、诊断站用户随后打开对应专著的比例,以及每年校验和失败的概率。
推导
# Chapter 2. p05: One update rule, three uses. A Beta(a, b) prior on a
# rate, after s successes in n trials, becomes Beta(a + s, b + n - s).
# (i) A checkout's conversion rate: prior Beta(2, 48), 30 sales in 500
# visits. (ii) The share of a diagnostic station's users who later open
# the matching book: prior Beta(1, 9), 12 of 80. (iii) The chance that
# a stored file fails its yearly checksum: prior Beta(1, 999), no
# failures in 2,000 checks. Posterior mean, and 95% interval from
# 200,000 draws. Seed 100.
import random
CASES = (("conversion", 2, 48, 30, 500), ("station to book", 1, 9, 12, 80),
("checksum failure", 1, 999, 0, 2000))
if __name__ == "__main__":
rng = random.Random(100)
print(" use prior data posterior 95% interval")
for name, a, b, s, n in CASES:
a2, b2 = a + s, b + n - s
xs = sorted(rng.betavariate(a2, b2) for _ in range(200000))
fmt = " {:<16} {:.4f} {:>4}/{:<5} {:9.5f} {:.5f} to {:.5f}"
print(fmt.format(name, a / (a + b), s, n, a2 / (a2 + b2),
xs[5000], xs[195000]))
# use prior data posterior 95% interval
# conversion 0.0400 30/500 0.05818 0.04024 to 0.07917
# station to book 0.1000 12/80 0.14444 0.07974 to 0.22355
# checksum failure 0.0010 0/2000 0.00033 0.00001 to 0.00123转化率的估计从先验均值 0.0400 移到 0.05818;诊断站到专著的比例从 0.1000 移到 0.14444,95% 区间很宽(0.07974 至 0.22355),因为只看到了 80 名用户;而 2,000 次干净的校验,给失败率留下每年 0.00123 的 95% 上限。规则相同;它能教给我们多少,只取决于每个领域提供了多少数据。
小结
风险率定律、对数求和指数的对偶与共轭更新,是本书库中反复出现的另外三块数学。每一块都可以原封不动地跨领域运行,而在每一块中,领域提供的是数据与参数,而不是形式。
参考文献
Fan, Y. (2026). Hundred-Volume Knowledge Hypergraphs, Multimodal Personal Prior Compilation and Autonomous Empire Scaling: Cross-Disciplinary Isomorphisms, Zero-Marginal-Cost Replication and Century-Scale Knowledge Preservation [百卷知识超图拓扑、多模态个人先验编译与自主知识帝国标度律] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090260 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).