What does Anti-Collapse Engineering with Two Anchors conclude, in numbers, about “Examining for Collapse: Why Inward-Looking Metrics Go Blind, and How Much Data an Outward-Looking Anchor Needs”?

Answer: Chapter 5 shows that inward-looking detection metrics grow more optimistic during collapse, the Good–Turing missing mass falling from 0.0677 to 0.0009 while the true loss rises.

Scope of the numbers: Parameters are illustrative; all calculations are generated by the book's programs, which call no model interface and use no reader's or customer's data. Marked boxes separate published evidence, runnable Python calculations (22 programs), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.0009, 0.0677

true support 500
genS_obsChao1GT_missingtrue_missing
0360.9467.90.06770.0661
1298.2351.00.04050.1090
2255.1288.50.02810.1425
5187.2200.50.01280.2095
10132.4137.30.00520.2861
2090.695.30.00300.3591
5047.047.60.00090.4792
Show the full program
# Chapter 5. p51: inward-looking detectors cannot see a lost tail.
# Model: K = 500 categories, Zipf p_i proportional to 1/i, N = 2000
# items per generation, each generation refitted on the counts of the
# one before (as in p23). On each generation's own sample we compute
# the Good-Turing missing mass f1/N (Good 1953) and the bias-corrected
# Chao1 richness S_obs + f1(f1-1)/(2(f2+1)) (Chao 1984), and compare
# them with the truth: the true mass of categories absent from the
# sample. R = 10 chains, seed fixed; means are shown.
import collections
import itertools
import math
import random

K, N, R = 500, 2000, 10
CHECK = (0, 1, 2, 5, 10, 20, 50)
H = math.fsum(1.0 / i for i in range(1, K + 1))
P = [1.0 / (i * H) for i in range(1, K + 1)]
rng = random.Random(20261114)
cats = list(range(K))
acc = collections.defaultdict(list)
for _ in range(R):
    cum = list(itertools.accumulate(P))
    for gen in range(max(CHECK) + 1):
        counts = collections.Counter(rng.choices(cats, cum_weights=cum,
                                                 k=N))
        if gen in CHECK:
            freq = collections.Counter(counts.values())
            f1, f2 = freq[1], freq[2]
            s_obs = len(counts)
            chao = s_obs + f1 * (f1 - 1) / (2.0 * (f2 + 1))
            miss = math.fsum(P[k] for k in cats if counts[k] == 0)
            acc[gen].append((s_obs, chao, f1 / N, miss))
        cum = list(itertools.accumulate(counts[k] for k in cats))
print("true support %d" % K)
print("gen  S_obs   Chao1   GT_missing  true_missing")
for gen in CHECK:
    m = [math.fsum(c) / R for c in zip(*acc[gen])]
    print("%-4d %-7.1f %-7.1f %-11.4f %.4f" % (gen, m[0], m[1], m[2],
                                              m[3]))

Single source and citation

Fan, Y. (2026). Anti-Collapse Engineering with Two Anchors of Truth: the Variance-Collapse Rate of Self-Generated Training Data and Immunity through Standard-Library Recomputation and Textual Collation (Version 1.0.0), Chapter 5: Examining for Collapse: Why Inward-Looking Metrics Go Blind, and How Much Data an Outward-Looking Anchor Needs. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-265 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026265,
  author    = {Fan, Yafu},
  title     = {Anti-Collapse Engineering with Two Anchors of Truth: the Variance-Collapse Rate of Self-Generated Training Data and Immunity through Standard-Library Recomputation and Textual Collation},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book