关于「一页一页地看:覆盖率的逐页波动与窄领域词表的边界」,《词汇覆盖率与速成边界》给出的定量结论是什么?
结论:第四章说明平均 98 % 的词表仍有 0.396 的页面低于 98 %,而一千六百个词的客服领域词表最多覆盖 0.7905。
数据口径:书中参数为示意,全部计算由书中程序生成;本书是非个人化的经营与教学参考,不是针对任何个人或企业的经营建议,也不承诺任何课程能在某个期限内带来流利。 书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.396, 98
| list | case | mean | sd | P10 | <95 % | <98 % |
| 4500 | independent | 0.9498 | 0.0126 | 0.9333 | 0.438 | 0.996 |
| 4500 | bursty k=4.0 | 0.9494 | 0.0280 | 0.9133 | 0.430 | 0.870 |
| 4500 | bursty k=1.0 | 0.9514 | 0.0496 | 0.8833 | 0.347 | 0.625 |
| 8000 | independent | 0.9800 | 0.0081 | 0.9700 | 0.001 | 0.396 |
| 8000 | bursty k=4.0 | 0.9798 | 0.0129 | 0.9633 | 0.021 | 0.389 |
| 8000 | bursty k=1.0 | 0.9807 | 0.0208 | 0.9533 | 0.079 | 0.323 |
查看完整验算程序
# Chapter 4. p41: an average of 98 % is not 98 % on every page
# Tokens on a 300-word page are drawn from the reference family curve of
# p23 (V = 20000, s = 2.389, q = 664.5). Case "independent": every
# token is an independent draw, so unknown words scatter like a
# binomial. Case "bursty k": each page also gets a topic factor G from
# a gamma law with mean 1 and shape k, and its rate of words beyond the
# list is multiplied by G, the way a page about a ferry or a refund
# repeats its own rare words. The mean is unchanged; the spread is not.
# The list is the top 4500 or 8000 families. Seeded; 2000 pages.
import math
import random
V, S, Q, PAGE, PAGES = 20000, 2.389, 664.5, 300, 2000
def weights():
return [(j + Q) ** (-S) for j in range(1, V + 1)]
def summary(covs):
covs = sorted(covs)
n = len(covs)
mean = math.fsum(covs) / n
sd = math.sqrt(math.fsum([(x - mean) ** 2 for x in covs]) / (n - 1))
p10 = covs[n // 10]
lo95 = len([x for x in covs if x < 0.95]) / n
lo98 = len([x for x in covs if x < 0.98]) / n
return mean, sd, p10, lo95, lo98
if __name__ == "__main__":
rng = random.Random(20261113)
w = weights()
toks = rng.choices(range(V), weights=w, k=PAGE * PAGES)
print("list case mean sd P10 <95 % <98 %")
for top in (4500, 8000):
covs = []
miss = []
for p in range(PAGES):
page = toks[p * PAGE:(p + 1) * PAGE]
out = len([t for t in page if t >= top])
covs.append(1 - out / PAGE)
miss.append(out / PAGE)
rate = math.fsum(miss) / PAGES
print(" {:4d} independent {:.4f} {:.4f} {:.4f} {:.3f}"
" {:.3f}".format(top, *summary(covs)))
for k in (4.0, 1.0):
covs = []
for p in range(PAGES):
g = rng.gammavariate(k, 1.0 / k)
pr = min(1.0, rate * g)
out = len([1 for i in range(PAGE) if rng.random() < pr])
covs.append(1 - out / PAGE)
print(" {:4d} bursty k={:3.1f} {:.4f} {:.4f} {:.4f}"
" {:.3f} {:.3f}".format(top, k, *summary(covs)))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《二语词汇覆盖率九十八阈值、齐普夫频率曲线与马来语闽南语速成边界母典》第 4 章「一页一页地看:覆盖率的逐页波动与窄领域词表的边界」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-228 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026228,
author = {Fan, Yafu},
title = {The 98% Lexical-Coverage Threshold, Zipfian Frequency Curves and Fast-Track Limits for Malay and Hokkien Learners},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}