关于「坍缩体检:向内看的指标为何失明,向外看的锚点要多少数据」,《抗模型崩溃双锚工程》给出的定量结论是什么?
结论:第五章说明向内看的检测指标在坍缩中越来越乐观,Good–Turing 缺失质量从 0.0677 降到 0.0009,而真实缺失在上升。
数据口径:书中参数为示意;全部计算由书中程序生成,不调用任何模型接口,不使用任何读者或客户的数据;本书是非个人化的经营与工程参考。 书中以标注框区分已发表的证据、可运行的 Python 计算(22 段程序)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0009, 0.0677
| true support 500 | ||||
| gen | S_obs | Chao1 | GT_missing | true_missing |
| 0 | 360.9 | 467.9 | 0.0677 | 0.0661 |
| 1 | 298.2 | 351.0 | 0.0405 | 0.1090 |
| 2 | 255.1 | 288.5 | 0.0281 | 0.1425 |
| 5 | 187.2 | 200.5 | 0.0128 | 0.2095 |
| 10 | 132.4 | 137.3 | 0.0052 | 0.2861 |
| 20 | 90.6 | 95.3 | 0.0030 | 0.3591 |
| 50 | 47.0 | 47.6 | 0.0009 | 0.4792 |
查看完整验算程序
# Chapter 5. p51: inward-looking detectors cannot see a lost tail.
# Model: K = 500 categories, Zipf p_i proportional to 1/i, N = 2000
# items per generation, each generation refitted on the counts of the
# one before (as in p23). On each generation's own sample we compute
# the Good-Turing missing mass f1/N (Good 1953) and the bias-corrected
# Chao1 richness S_obs + f1(f1-1)/(2(f2+1)) (Chao 1984), and compare
# them with the truth: the true mass of categories absent from the
# sample. R = 10 chains, seed fixed; means are shown.
import collections
import itertools
import math
import random
K, N, R = 500, 2000, 10
CHECK = (0, 1, 2, 5, 10, 20, 50)
H = math.fsum(1.0 / i for i in range(1, K + 1))
P = [1.0 / (i * H) for i in range(1, K + 1)]
rng = random.Random(20261114)
cats = list(range(K))
acc = collections.defaultdict(list)
for _ in range(R):
cum = list(itertools.accumulate(P))
for gen in range(max(CHECK) + 1):
counts = collections.Counter(rng.choices(cats, cum_weights=cum,
k=N))
if gen in CHECK:
freq = collections.Counter(counts.values())
f1, f2 = freq[1], freq[2]
s_obs = len(counts)
chao = s_obs + f1 * (f1 - 1) / (2.0 * (f2 + 1))
miss = math.fsum(P[k] for k in cats if counts[k] == 0)
acc[gen].append((s_obs, chao, f1 / N, miss))
cum = list(itertools.accumulate(counts[k] for k in cats))
print("true support %d" % K)
print("gen S_obs Chao1 GT_missing true_missing")
for gen in CHECK:
m = [math.fsum(c) / R for c in zip(*acc[gen])]
print("%-4d %-7.1f %-7.1f %-11.4f %.4f" % (gen, m[0], m[1], m[2],
m[3]))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《抗模型崩溃双锚真源工程、自产语料回喂方差坍缩速率与标准库验算校勘免疫母典》第 5 章「坍缩体检:向内看的指标为何失明,向外看的锚点要多少数据」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-265 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026265,
author = {Fan, Yafu},
title = {Anti-Collapse Engineering with Two Anchors of Truth: the Variance-Collapse Rate of Self-Generated Training Data and Immunity through Standard-Library Recomputation and Textual Collation},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}