关于「坍缩体检:向内看的指标为何失明,向外看的锚点要多少数据」,《抗模型崩溃双锚工程》给出的定量结论是什么?

结论:第五章说明向内看的检测指标在坍缩中越来越乐观,Good–Turing 缺失质量从 0.0677 降到 0.0009,而真实缺失在上升。

数据口径:书中参数为示意;全部计算由书中程序生成,不调用任何模型接口,不使用任何读者或客户的数据;本书是非个人化的经营与工程参考。 书中以标注框区分已发表的证据、可运行的 Python 计算(22 段程序)、原典与作者的假说。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.0009, 0.0677

true support 500
genS_obsChao1GT_missingtrue_missing
0360.9467.90.06770.0661
1298.2351.00.04050.1090
2255.1288.50.02810.1425
5187.2200.50.01280.2095
10132.4137.30.00520.2861
2090.695.30.00300.3591
5047.047.60.00090.4792
查看完整验算程序
# Chapter 5. p51: inward-looking detectors cannot see a lost tail.
# Model: K = 500 categories, Zipf p_i proportional to 1/i, N = 2000
# items per generation, each generation refitted on the counts of the
# one before (as in p23). On each generation's own sample we compute
# the Good-Turing missing mass f1/N (Good 1953) and the bias-corrected
# Chao1 richness S_obs + f1(f1-1)/(2(f2+1)) (Chao 1984), and compare
# them with the truth: the true mass of categories absent from the
# sample. R = 10 chains, seed fixed; means are shown.
import collections
import itertools
import math
import random

K, N, R = 500, 2000, 10
CHECK = (0, 1, 2, 5, 10, 20, 50)
H = math.fsum(1.0 / i for i in range(1, K + 1))
P = [1.0 / (i * H) for i in range(1, K + 1)]
rng = random.Random(20261114)
cats = list(range(K))
acc = collections.defaultdict(list)
for _ in range(R):
    cum = list(itertools.accumulate(P))
    for gen in range(max(CHECK) + 1):
        counts = collections.Counter(rng.choices(cats, cum_weights=cum,
                                                 k=N))
        if gen in CHECK:
            freq = collections.Counter(counts.values())
            f1, f2 = freq[1], freq[2]
            s_obs = len(counts)
            chao = s_obs + f1 * (f1 - 1) / (2.0 * (f2 + 1))
            miss = math.fsum(P[k] for k in cats if counts[k] == 0)
            acc[gen].append((s_obs, chao, f1 / N, miss))
        cum = list(itertools.accumulate(counts[k] for k in cats))
print("true support %d" % K)
print("gen  S_obs   Chao1   GT_missing  true_missing")
for gen in CHECK:
    m = [math.fsum(c) / R for c in zip(*acc[gen])]
    print("%-4d %-7.1f %-7.1f %-11.4f %.4f" % (gen, m[0], m[1], m[2],
                                              m[3]))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《抗模型崩溃双锚真源工程、自产语料回喂方差坍缩速率与标准库验算校勘免疫母典》第 5 章「坍缩体检:向内看的指标为何失明,向外看的锚点要多少数据」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-265 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026265,
  author    = {Fan, Yafu},
  title     = {Anti-Collapse Engineering with Two Anchors of Truth: the Variance-Collapse Rate of Self-Generated Training Data and Immunity through Standard-Library Recomputation and Textual Collation},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论