关于「反方证据:有限样本、学来的压缩与判别力的计分方式」,《护栏压缩的极限》给出的定量结论是什么?
结论:第六章陈列反方证据:50 份样本时 0.920 的重复实验选错拐点,从少量记录学出的六条规则只保留 0.017 比特。
数据口径:本书把候选规则写成离散信源、把审稿裁决写成相关变量,用香农的率失真函数、布拉赫特与有本的迭代和提斯比等人的信息瓶颈逐条检验,并修正蓝图:原文没有「断崖」之说;最优曲线是凹的、连续的,越压越陡,示意信源上每删一比特的代价从 0.012 升到 0.592;断崖只在整数条数低于需要区分的裁决组数、解的结构跨过临界点、或用离散计分去量时出现;审稿人不犹豫时代价恰是直线。 书中参数为示意;全部计算由书中程序生成;本书是非个人化经营建议,不构成法律或内容合规意见,任何护栏规则的删并与上线须由主事者人工复核放行。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.920, 50
| true I(X;Y) = 0.7534 bits, true knee = 3 rules | ||||||
| n | plug-in | bias | 10.1/n | Miller-Madow | knee!=3 | mean knee |
| 50 | 0.9075 | +0.1541 | 0.2020 | 0.7055 | 0.920 | 4.28 |
| 100 | 0.8366 | +0.0832 | 0.1010 | 0.7356 | 0.920 | 4.20 |
| 200 | 0.8083 | +0.0549 | 0.0505 | 0.7578 | 0.810 | 3.96 |
| 400 | 0.7786 | +0.0252 | 0.0252 | 0.7533 | 0.780 | 3.81 |
| 1600 | 0.7600 | +0.0067 | 0.0063 | 0.7537 | 0.280 | 3.28 |
查看完整验算程序
# Chapter 6. p61: the curve you measure is not the curve that exists.
# Draw n labelled drafts from the guardrail source of p11, estimate
# I(X;Y) by plug-in, and pick the knee (fewest agglomerative rules that
# keep 95 percent of the estimated information). Plug-in information is
# biased upwards by about (|X| - 1)(|Y| - 1) / (2 n ln 2) bits (Miller-
# Madow; Paninski 2003); the knee moves with it. 200 seeded repeats.
import math
import random
PX = [0.40, 0.18, 0.10, 0.07, 0.10, 0.06, 0.05, 0.04]
PYX = [[0.92, 0.06, 0.02], [0.25, 0.70, 0.05], [0.30, 0.55, 0.15],
[0.10, 0.60, 0.30], [0.05, 0.15, 0.80], [0.03, 0.12, 0.85],
[0.02, 0.08, 0.90], [0.01, 0.04, 0.95]]
CELLS = [(x, y) for x in range(8) for y in range(3)]
WEIGHTS = [PX[x] * PYX[x][y] for x, y in CELLS]
def mutual(groups):
tot = math.fsum(math.fsum(g) for g in groups)
py = [math.fsum(g[y] for g in groups) / tot for y in range(3)]
terms = []
for g in groups:
pg = math.fsum(g) / tot
for y in range(3):
v = g[y] / tot
if v > 0:
terms.append(v * math.log2(v / (pg * py[y])))
return math.fsum(terms)
def knee(groups, share=0.95):
full = mutual(groups)
groups = [list(g) for g in groups if math.fsum(g) > 0]
best_k = len(groups)
while len(groups) > 1:
cand = None
for i in range(len(groups)):
for j in range(i + 1, len(groups)):
m = [a + b for a, b in zip(groups[i], groups[j])]
rest = [g for k, g in enumerate(groups) if k not in (i, j)]
v = mutual(rest + [m])
if cand is None or v > cand[0]:
cand = (v, rest + [m])
groups = cand[1]
if cand[0] >= share * full:
best_k = len(groups)
return best_k
true_groups = [[PX[x] * PYX[x][y] for y in range(3)] for x in range(8)]
true_i = mutual(true_groups)
print("true I(X;Y) = %.4f bits, true knee = %d rules"
% (true_i, knee(true_groups)))
rng = random.Random(20261113)
print(" n plug-in bias 10.1/n Miller-Madow knee!=3 mean knee")
for n in [50, 100, 200, 400, 1600]:
est, mm, wrong, ks = [], [], 0, []
for _ in range(200):
counts = [[0.0] * 3 for _ in range(8)]
for x, y in rng.choices(CELLS, weights=WEIGHTS, k=n):
counts[x][y] += 1.0
v = mutual(counts)
est.append(v)
mm.append(v - 7.0 / (n * math.log(2)))
k = knee(counts)
ks.append(float(k))
if k != 3:
wrong += 1
mean = math.fsum(est) / 200
print(" %4d %6.4f %+6.4f %6.4f %6.4f %5.3f %4.2f"
% (n, mean, mean - true_i, 7.0 / (n * math.log(2)),
math.fsum(mm) / 200, wrong / 200, math.fsum(ks) / 200))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《信息瓶颈率失真极值、双语专著到全局护栏的极限压缩比与判别力保真母典》第 6 章「反方证据:有限样本、学来的压缩与判别力的计分方式」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-250 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026250,
author = {Fan, Yafu},
title = {The Information Bottleneck and Rate-Distortion Limits: How Far a Bilingual Monograph Can Be Compressed into Guardrails before Discriminative Power Falls Away},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}