关于「实证闭环:客服话术词表、团队学习计划与推翻本书的方案」,《词汇覆盖率与速成边界》给出的定量结论是什么?
结论:第八章给出马来西亚与台湾客服团队的建表、测量与复测流程,蒸馏小工具「词表覆盖率计算器」(示范中词缀规则把覆盖率从 0.6667 提到 0.8667),以及三项预注册检验与推翻本书核心命题的条件。书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。
数据口径:书中参数为示意,全部计算由书中程序生成;本书是非个人化的经营与教学参考,不是针对任何个人或企业的经营建议,也不承诺任何课程能在某个期限内带来流利。 书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.6667, 0.8667
Malay, affix rules off: tokens 30, known 20, coverage 0.6667
unknown: dibuat(1) dihantar(1) menghubungi(1) menunggu(1)
pembayaran(1) penghantaran(1) pesanan(1) sebentar(1) talian(1)
termasuk(1)
unknown per 300-word page: 100.0
list multiplier for 0.95: x 5.7
list multiplier for 0.98: x 10.2
Malay, affix rules on: tokens 30, known 26, coverage 0.8667
unknown: menghubungi(1) sebentar(1) talian(1) termasuk(1)
unknown per 300-word page: 40.0
list multiplier for 0.95: x 2.2
list multiplier for 0.98: x 3.9
Hokkien (Tai-lo, tone numbers): tokens 10, known 8, coverage 0.8000
unknown: gua7-tse7(1) phainn2-se3(1)
unknown per 300-word page: 60.0
list multiplier for 0.95: x 3.2
list multiplier for 0.98: x 5.7查看完整验算程序
# Chapter 8. p82: the core of the word-list coverage calculator
# Tokens are runs of letters and digits, hyphenated runs kept whole, so
# Tai-lo with tone numbers (to1-sia7) counts as one word. A token is
# known if it is on the list or, in Malay mode, if the affix rules of
# p32 reduce it to a root on the list. Coverage = known / tokens. The
# projection finds the list size n_eq on the reference curve of p23
# (V = 20000, s = 2.389, q = 664.5) that gives the measured coverage
# and scales the list by n_target / n_eq; it is a rough planning
# figure, not a measurement. Demo texts and lists are hand-made.
import math
import re
import textwrap
from collections import Counter
V, S, Q = 20000, 2.389, 664.5
PREFIX = [("meny", "s"), ("meng", ""), ("meng", "k"), ("mem", ""),
("mem", "p"), ("men", ""), ("men", "t"), ("me", ""),
("peny", "s"), ("peng", ""), ("peng", "k"), ("pem", ""),
("pem", "p"), ("pen", ""), ("pen", "t"), ("pel", ""),
("per", ""), ("pe", ""), ("bel", ""), ("ber", ""), ("di", ""),
("ter", ""), ("", "")]
SUFFIX = ["kan", "an", "i", ""]
def tokens(text):
return re.findall(r"[a-z0-9]+(?:-[a-z0-9]+)*", text.lower())
def malay_root_known(word, known):
for pre, restore in PREFIX:
if word.startswith(pre):
stem = restore + word[len(pre):]
for suf in SUFFIX:
if suf == "" or stem.endswith(suf):
root = stem[:len(stem) - len(suf)] if suf else stem
if root in known:
return True
return False
def coverage(text, wordlist, malay=False):
known = set(tokens(wordlist))
toks = tokens(text)
miss = Counter()
for t in toks:
if t in known or (malay and malay_root_known(t, known)):
continue
miss[t] += 1
return len(toks), len(toks) - sum_counts(miss), miss
def sum_counts(counter):
n = 0
for v in counter.values():
n += v
return n
CURVE = [(r + Q) ** (-S) for r in range(1, V + 1)]
TOTAL = math.fsum(CURVE)
def n_for(c):
lo, hi = 0, V
while hi - lo > 1:
mid = (lo + hi) // 2
if math.fsum(CURVE[:mid]) / TOTAL >= c:
hi = mid
else:
lo = mid
return hi
def report(name, text, wordlist, malay):
n, k, miss = coverage(text, wordlist, malay)
c = k / n
print("{}: tokens {}, known {}, coverage {:.4f}".format(name, n, k, c))
words = " ".join("{}({})".format(w, miss[w]) for w in sorted(miss))
print(textwrap.fill("unknown: " + words, width=72,
initial_indent=" ", subsequent_indent=" "))
print(" unknown per 300-word page: {:.1f}".format(300 * (1 - c)))
neq = n_for(c)
for tgt in (0.95, 0.98):
print(" list multiplier for {:.2f}: x {:.1f}".format(
tgt, n_for(tgt) / neq))
MS_TEXT = ("Selamat pagi, boleh saya bantu? Terima kasih kerana menunggu."
" Pesanan anda sudah dihantar. Harga ini termasuk penghantaran."
" Sila tunggu sebentar. Kami akan menghubungi anda esok."
" Pembayaran boleh dibuat dalam talian.")
MS_LIST = ("selamat pagi boleh saya bantu terima kasih kerana tunggu "
"pesan anda sudah hantar harga ini sila kami akan esok bayar "
"buat dalam")
HOK_TEXT = ("li2 ho2. to1-sia7. phainn2-se3. tsit4 e5 gua7-tse7 tsinn5?"
" u7 bo5?")
HOK_LIST = "li2 ho2 to1-sia7 tsit4 e5 tsinn5 u7 bo5"
if __name__ == "__main__":
report("Malay, affix rules off", MS_TEXT, MS_LIST, False)
report("Malay, affix rules on", MS_TEXT, MS_LIST, True)
report("Hokkien (Tai-lo, tone numbers)", HOK_TEXT, HOK_LIST, False)唯一数据源与引用
范雅福(Fan, Y.). (2026). 《二语词汇覆盖率九十八阈值、齐普夫频率曲线与马来语闽南语速成边界母典》第 8 章「实证闭环:客服话术词表、团队学习计划与推翻本书的方案」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-228 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026228,
author = {Fan, Yafu},
title = {The 98% Lexical-Coverage Threshold, Zipfian Frequency Curves and Fast-Track Limits for Malay and Hokkien Learners},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}