关于「实证闭环:客服话术词表、团队学习计划与推翻本书的方案」,《词汇覆盖率与速成边界》给出的定量结论是什么?

结论:第八章给出马来西亚与台湾客服团队的建表、测量与复测流程,蒸馏小工具「词表覆盖率计算器」(示范中词缀规则把覆盖率从 0.6667 提到 0.8667),以及三项预注册检验与推翻本书核心命题的条件。书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。

数据口径:书中参数为示意,全部计算由书中程序生成;本书是非个人化的经营与教学参考,不是针对任何个人或企业的经营建议,也不承诺任何课程能在某个期限内带来流利。 书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.6667, 0.8667

Malay, affix rules off: tokens 30, known 20, coverage 0.6667
  unknown: dibuat(1) dihantar(1) menghubungi(1) menunggu(1)
    pembayaran(1) penghantaran(1) pesanan(1) sebentar(1) talian(1)
    termasuk(1)
  unknown per 300-word page: 100.0
  list multiplier for 0.95: x 5.7
  list multiplier for 0.98: x 10.2
Malay, affix rules on: tokens 30, known 26, coverage 0.8667
  unknown: menghubungi(1) sebentar(1) talian(1) termasuk(1)
  unknown per 300-word page: 40.0
  list multiplier for 0.95: x 2.2
  list multiplier for 0.98: x 3.9
Hokkien (Tai-lo, tone numbers): tokens 10, known 8, coverage 0.8000
  unknown: gua7-tse7(1) phainn2-se3(1)
  unknown per 300-word page: 60.0
  list multiplier for 0.95: x 3.2
  list multiplier for 0.98: x 5.7
查看完整验算程序
# Chapter 8. p82: the core of the word-list coverage calculator
# Tokens are runs of letters and digits, hyphenated runs kept whole, so
# Tai-lo with tone numbers (to1-sia7) counts as one word. A token is
# known if it is on the list or, in Malay mode, if the affix rules of
# p32 reduce it to a root on the list. Coverage = known / tokens. The
# projection finds the list size n_eq on the reference curve of p23
# (V = 20000, s = 2.389, q = 664.5) that gives the measured coverage
# and scales the list by n_target / n_eq; it is a rough planning
# figure, not a measurement. Demo texts and lists are hand-made.
import math
import re
import textwrap
from collections import Counter

V, S, Q = 20000, 2.389, 664.5
PREFIX = [("meny", "s"), ("meng", ""), ("meng", "k"), ("mem", ""),
          ("mem", "p"), ("men", ""), ("men", "t"), ("me", ""),
          ("peny", "s"), ("peng", ""), ("peng", "k"), ("pem", ""),
          ("pem", "p"), ("pen", ""), ("pen", "t"), ("pel", ""),
          ("per", ""), ("pe", ""), ("bel", ""), ("ber", ""), ("di", ""),
          ("ter", ""), ("", "")]
SUFFIX = ["kan", "an", "i", ""]


def tokens(text):
    return re.findall(r"[a-z0-9]+(?:-[a-z0-9]+)*", text.lower())


def malay_root_known(word, known):
    for pre, restore in PREFIX:
        if word.startswith(pre):
            stem = restore + word[len(pre):]
            for suf in SUFFIX:
                if suf == "" or stem.endswith(suf):
                    root = stem[:len(stem) - len(suf)] if suf else stem
                    if root in known:
                        return True
    return False


def coverage(text, wordlist, malay=False):
    known = set(tokens(wordlist))
    toks = tokens(text)
    miss = Counter()
    for t in toks:
        if t in known or (malay and malay_root_known(t, known)):
            continue
        miss[t] += 1
    return len(toks), len(toks) - sum_counts(miss), miss


def sum_counts(counter):
    n = 0
    for v in counter.values():
        n += v
    return n


CURVE = [(r + Q) ** (-S) for r in range(1, V + 1)]
TOTAL = math.fsum(CURVE)


def n_for(c):
    lo, hi = 0, V
    while hi - lo > 1:
        mid = (lo + hi) // 2
        if math.fsum(CURVE[:mid]) / TOTAL >= c:
            hi = mid
        else:
            lo = mid
    return hi


def report(name, text, wordlist, malay):
    n, k, miss = coverage(text, wordlist, malay)
    c = k / n
    print("{}: tokens {}, known {}, coverage {:.4f}".format(name, n, k, c))
    words = " ".join("{}({})".format(w, miss[w]) for w in sorted(miss))
    print(textwrap.fill("unknown: " + words, width=72,
                        initial_indent="  ", subsequent_indent="    "))
    print("  unknown per 300-word page: {:.1f}".format(300 * (1 - c)))
    neq = n_for(c)
    for tgt in (0.95, 0.98):
        print("  list multiplier for {:.2f}: x {:.1f}".format(
            tgt, n_for(tgt) / neq))


MS_TEXT = ("Selamat pagi, boleh saya bantu? Terima kasih kerana menunggu."
           " Pesanan anda sudah dihantar. Harga ini termasuk penghantaran."
           " Sila tunggu sebentar. Kami akan menghubungi anda esok."
           " Pembayaran boleh dibuat dalam talian.")
MS_LIST = ("selamat pagi boleh saya bantu terima kasih kerana tunggu "
           "pesan anda sudah hantar harga ini sila kami akan esok bayar "
           "buat dalam")
HOK_TEXT = ("li2 ho2. to1-sia7. phainn2-se3. tsit4 e5 gua7-tse7 tsinn5?"
            " u7 bo5?")
HOK_LIST = "li2 ho2 to1-sia7 tsit4 e5 tsinn5 u7 bo5"

if __name__ == "__main__":
    report("Malay, affix rules off", MS_TEXT, MS_LIST, False)
    report("Malay, affix rules on", MS_TEXT, MS_LIST, True)
    report("Hokkien (Tai-lo, tone numbers)", HOK_TEXT, HOK_LIST, False)

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《二语词汇覆盖率九十八阈值、齐普夫频率曲线与马来语闽南语速成边界母典》第 8 章「实证闭环:客服话术词表、团队学习计划与推翻本书的方案」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-228 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026228,
  author    = {Fan, Yafu},
  title     = {The 98% Lexical-Coverage Threshold, Zipfian Frequency Curves and Fast-Track Limits for Malay and Hokkien Learners},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论