关于「一页一页地看:覆盖率的逐页波动与窄领域词表的边界」,《词汇覆盖率与速成边界》给出的定量结论是什么?

结论:第四章说明平均 98 % 的词表仍有 0.396 的页面低于 98 %,而一千六百个词的客服领域词表最多覆盖 0.7905。

数据口径:书中参数为示意,全部计算由书中程序生成;本书是非个人化的经营与教学参考,不是针对任何个人或企业的经营建议,也不承诺任何课程能在某个期限内带来流利。 书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.396, 98

listcasemeansdP10<95 %<98 %
4500independent0.94980.01260.93330.4380.996
4500bursty k=4.00.94940.02800.91330.4300.870
4500bursty k=1.00.95140.04960.88330.3470.625
8000independent0.98000.00810.97000.0010.396
8000bursty k=4.00.97980.01290.96330.0210.389
8000bursty k=1.00.98070.02080.95330.0790.323
查看完整验算程序
# Chapter 4. p41: an average of 98 % is not 98 % on every page
# Tokens on a 300-word page are drawn from the reference family curve of
# p23 (V = 20000, s = 2.389, q = 664.5). Case "independent": every
# token is an independent draw, so unknown words scatter like a
# binomial. Case "bursty k": each page also gets a topic factor G from
# a gamma law with mean 1 and shape k, and its rate of words beyond the
# list is multiplied by G, the way a page about a ferry or a refund
# repeats its own rare words. The mean is unchanged; the spread is not.
# The list is the top 4500 or 8000 families. Seeded; 2000 pages.
import math
import random

V, S, Q, PAGE, PAGES = 20000, 2.389, 664.5, 300, 2000


def weights():
    return [(j + Q) ** (-S) for j in range(1, V + 1)]


def summary(covs):
    covs = sorted(covs)
    n = len(covs)
    mean = math.fsum(covs) / n
    sd = math.sqrt(math.fsum([(x - mean) ** 2 for x in covs]) / (n - 1))
    p10 = covs[n // 10]
    lo95 = len([x for x in covs if x < 0.95]) / n
    lo98 = len([x for x in covs if x < 0.98]) / n
    return mean, sd, p10, lo95, lo98


if __name__ == "__main__":
    rng = random.Random(20261113)
    w = weights()
    toks = rng.choices(range(V), weights=w, k=PAGE * PAGES)
    print("list   case           mean    sd     P10   <95 %   <98 %")
    for top in (4500, 8000):
        covs = []
        miss = []
        for p in range(PAGES):
            page = toks[p * PAGE:(p + 1) * PAGE]
            out = len([t for t in page if t >= top])
            covs.append(1 - out / PAGE)
            miss.append(out / PAGE)
        rate = math.fsum(miss) / PAGES
        print("  {:4d}  independent  {:.4f}  {:.4f}  {:.4f}  {:.3f}"
              "  {:.3f}".format(top, *summary(covs)))
        for k in (4.0, 1.0):
            covs = []
            for p in range(PAGES):
                g = rng.gammavariate(k, 1.0 / k)
                pr = min(1.0, rate * g)
                out = len([1 for i in range(PAGE) if rng.random() < pr])
                covs.append(1 - out / PAGE)
            print("  {:4d}  bursty k={:3.1f}  {:.4f}  {:.4f}  {:.4f}"
                  "  {:.3f}  {:.3f}".format(top, k, *summary(covs)))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《二语词汇覆盖率九十八阈值、齐普夫频率曲线与马来语闽南语速成边界母典》第 4 章「一页一页地看:覆盖率的逐页波动与窄领域词表的边界」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-228 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026228,
  author    = {Fan, Yafu},
  title     = {The 98% Lexical-Coverage Threshold, Zipfian Frequency Curves and Fast-Track Limits for Malay and Hokkien Learners},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论