What does Lexical Coverage and the Limits of Fast Tracks conclude, in numbers, about “Page by Page: How Coverage Varies Across Pages and Where a Narrow Domain List Stops”?

Answer: Chapter 4 shows that a list averaging 98 % still leaves 0.396 of pages below 98 %, and that a customer-service domain list of sixteen hundred items covers at most 0.7905.

Scope of the numbers: Its parameters are illustrative and all calculations are generated by its programs. It is general reference material for owners and teachers, not business advice for any person or firm, and it promises no fluency from any course within any period.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.396, 98

listcasemeansdP10<95 %<98 %
4500independent0.94980.01260.93330.4380.996
4500bursty k=4.00.94940.02800.91330.4300.870
4500bursty k=1.00.95140.04960.88330.3470.625
8000independent0.98000.00810.97000.0010.396
8000bursty k=4.00.97980.01290.96330.0210.389
8000bursty k=1.00.98070.02080.95330.0790.323
Show the full program
# Chapter 4. p41: an average of 98 % is not 98 % on every page
# Tokens on a 300-word page are drawn from the reference family curve of
# p23 (V = 20000, s = 2.389, q = 664.5). Case "independent": every
# token is an independent draw, so unknown words scatter like a
# binomial. Case "bursty k": each page also gets a topic factor G from
# a gamma law with mean 1 and shape k, and its rate of words beyond the
# list is multiplied by G, the way a page about a ferry or a refund
# repeats its own rare words. The mean is unchanged; the spread is not.
# The list is the top 4500 or 8000 families. Seeded; 2000 pages.
import math
import random

V, S, Q, PAGE, PAGES = 20000, 2.389, 664.5, 300, 2000


def weights():
    return [(j + Q) ** (-S) for j in range(1, V + 1)]


def summary(covs):
    covs = sorted(covs)
    n = len(covs)
    mean = math.fsum(covs) / n
    sd = math.sqrt(math.fsum([(x - mean) ** 2 for x in covs]) / (n - 1))
    p10 = covs[n // 10]
    lo95 = len([x for x in covs if x < 0.95]) / n
    lo98 = len([x for x in covs if x < 0.98]) / n
    return mean, sd, p10, lo95, lo98


if __name__ == "__main__":
    rng = random.Random(20261113)
    w = weights()
    toks = rng.choices(range(V), weights=w, k=PAGE * PAGES)
    print("list   case           mean    sd     P10   <95 %   <98 %")
    for top in (4500, 8000):
        covs = []
        miss = []
        for p in range(PAGES):
            page = toks[p * PAGE:(p + 1) * PAGE]
            out = len([t for t in page if t >= top])
            covs.append(1 - out / PAGE)
            miss.append(out / PAGE)
        rate = math.fsum(miss) / PAGES
        print("  {:4d}  independent  {:.4f}  {:.4f}  {:.4f}  {:.3f}"
              "  {:.3f}".format(top, *summary(covs)))
        for k in (4.0, 1.0):
            covs = []
            for p in range(PAGES):
                g = rng.gammavariate(k, 1.0 / k)
                pr = min(1.0, rate * g)
                out = len([1 for i in range(PAGE) if rng.random() < pr])
                covs.append(1 - out / PAGE)
            print("  {:4d}  bursty k={:3.1f}  {:.4f}  {:.4f}  {:.4f}"
                  "  {:.3f}  {:.3f}".format(top, k, *summary(covs)))

Single source and citation

Fan, Y. (2026). The 98% Lexical-Coverage Threshold, Zipfian Frequency Curves and Fast-Track Limits for Malay and Hokkien Learners (Version 1.0.0), Chapter 4: Page by Page: How Coverage Varies Across Pages and Where a Narrow Domain List Stops. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-228 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026228,
  author    = {Fan, Yafu},
  title     = {The 98% Lexical-Coverage Threshold, Zipfian Frequency Curves and Fast-Track Limits for Malay and Hokkien Learners},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book