What does Lexical Coverage and the Limits of Fast Tracks conclude, in numbers, about “How to Count a Word: Forms, Lemmas, Families and Malay Affixes”?

Answer: Between "two thousand words" and "eight thousand families" lies a unit of counting that is usually left out. p31 shows that on the same reference curve the forms needed for 98 % can be between 1.95 and 4.62 times the families; and a 98 % counted in families falls to between 0.7389 and 0.8167 when a learner has only an even chance of recognising derived forms. Malay is rich in affixes, and p32 uses a rule set that accepts only lexicon roots to assign thirty-five common forms to sixteen families; Hokkien has few affixes, the difficulty moves to segmentation, and the Tâi-lô hyphen is what makes counting repeatable. The next chapter leaves averages behind and looks at text page by page: what does an average coverage of 98 % look like on each single page?

Scope of the numbers: Its parameters are illustrative and all calculations are generated by its programs. It is general reference material for owners and teachers, not business advice for any person or firm, and it promises no fluency from any course within any period.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.7389, 0.8167, 1.95, 4.62, 98

families for 98 %: 7998
members mword forms for 95 %for 98 %forms/families
1449979981.00
28748155781.95
415775282673.53
620395369194.62
top 8000 families, derived forms recognised with prob rho
m = 2:rho 1.0 -> 0.9800rho 0.8 -> 0.9147rho 0.5 -> 0.8167
m = 4:rho 1.0 -> 0.9800rho 0.8 -> 0.8885rho 0.5 -> 0.7513
m = 6:rho 1.0 -> 0.9800rho 0.8 -> 0.8836rho 0.5 -> 0.7389
Show the full program
# Chapter 3. p31: the same text counted in families and in word forms
# Model: F = 20000 word families with frequencies (j + q)^(-s), using
# the reference fit of p23 (s = 2.389, q = 664.5). Each family has m
# members whose shares fall by half from one member to the next (head,
# then derived and inflected forms). We count how many families and
# how many separate word forms a 98 % list needs, and how much a list
# of the top 8000 families covers when a learner recognises a derived
# form of a known family only with probability rho. The member counts
# and rho are illustrative: m = 1 for an analytic language with little
# affixation, larger m for languages such as Malay that build families
# with productive prefixes and suffixes.
import math

F, S, Q = 20000, 2.389, 664.5


def family_freqs():
    raw = [(j + Q) ** (-S) for j in range(1, F + 1)]
    tot = math.fsum(raw)
    return [x / tot for x in raw]


def shares(m):
    raw = [0.5 ** i for i in range(m)]
    tot = math.fsum(raw)
    return [x / tot for x in raw]


def smallest_list(freqs, c):
    """Smallest n with the n largest values summing to at least c."""
    srt = sorted(freqs, reverse=True)
    lo, hi = 0, len(srt)
    while hi - lo > 1:
        mid = (lo + hi) // 2
        if math.fsum(srt[:mid]) >= c:
            hi = mid
        else:
            lo = mid
    return hi


if __name__ == "__main__":
    fam = family_freqs()
    print("families for 98 %: {}".format(smallest_list(fam, 0.98)))
    print("members m  word forms for 95 %  for 98 %  forms/families")
    n98f = smallest_list(fam, 0.98)
    for m in (1, 2, 4, 6):
        w = shares(m)
        forms = [f * x for f in fam for x in w]
        n95, n98 = smallest_list(forms, 0.95), smallest_list(forms, 0.98)
        print("  {:7d}  {:19d}  {:8d}  {:14.2f}".format(
            m, n95, n98, n98 / n98f))
    print("top 8000 families, derived forms recognised with prob rho")
    known = math.fsum(fam[:8000])
    for m in (2, 4, 6):
        head = shares(m)[0]
        row = ["  m = {}:".format(m)]
        for rho in (1.0, 0.8, 0.5):
            row.append("rho {:.1f} -> {:.4f}".format(
                rho, known * (head + rho * (1 - head))))
        print("  ".join(row))

Single source and citation

Fan, Y. (2026). The 98% Lexical-Coverage Threshold, Zipfian Frequency Curves and Fast-Track Limits for Malay and Hokkien Learners (Version 1.0.0), Chapter 3: How to Count a Word: Forms, Lemmas, Families and Malay Affixes. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-228 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026228,
  author    = {Fan, Yafu},
  title     = {The 98% Lexical-Coverage Threshold, Zipfian Frequency Curves and Fast-Track Limits for Malay and Hokkien Learners},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book