What does Chinese Readability conclude, in numbers, about “The Long Tail of Character Frequency: the General Standard Table, a Zipf Model and an Illustrative Frequency-Tier Table”?

Answer: The program first checks that the three levels add up: 3500 plus 3000 plus 1605 equals 8105. The exponent that meets the target coverage of 0.99 is s = 1.3840. In this model the first 1000 characters cover 0.9676, the first 2500 cover 0.9850 and level one covers 0.9900, so 1.00 character in every hundred falls outside level one; levels one and two together, 6500 characters, cover 0.9977.

Scope of the numbers: Parameters are illustrative; all calculations are generated by the book's programs; the book offers general rather than individualised business advice. Chapters 2 and 3 treat common-character lists, an illustrative frequency-tier table and the rules for cutting sentences.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.9676, 0.9850, 0.99, 0.9900, 0.9977, 1.00

levels 3500 + 3000 + 1605 = 8105
Zipf exponent giving level-one coverage 0.99: s = 1.3840
top kcoverageoutside per 100 hanzi
1000.884411.56
5000.94975.03
10000.96763.24
25000.98501.50
35000.99001.00
65000.99770.23
81051.00000.00
chance a sentence holds >= 1 hanzi outside the common set
Lc=0.99c=0.95
40.03940.1855
80.07730.3366
160.14850.5599
250.22220.7226
370.31060.8501
720.51500.9751
Show the full program
# Chapter 2. p21: how much text the 3500 level-one hanzi cover, in a toy
# Zipf model. Facts: the Table of General Standard Chinese Characters
# (State Council, 2013) lists 8105 hanzi in three levels of 3500, 3000
# and 1605. Model (illustrative): token share of rank r is proportional
# to r^-s over the closed set of 8105; s is solved so that level one
# covers a target share C1 = 0.99 (an assumed round figure, not a
# measured one). A sentence of L independent hanzi then contains at
# least one hanzi outside level one with chance 1 - c^L.
import math

N, LEVELS, C1 = 8105, (3500, 3000, 1605), 0.99


def harmonic(k, s):
    return math.fsum(r ** -s for r in range(1, k + 1))


def coverage(k, s):
    return harmonic(k, s) / harmonic(N, s)


def solve_s(target, k):
    lo, hi = 0.5, 3.0
    for _ in range(60):
        mid = 0.5 * (lo + hi)
        if coverage(k, mid) < target:
            lo = mid
        else:
            hi = mid
    return 0.5 * (lo + hi)


if __name__ == "__main__":
    print("levels {} + {} + {} = {}".format(*LEVELS, N))
    s = solve_s(C1, LEVELS[0])
    print("Zipf exponent giving level-one coverage {:.2f}: s = {:.4f}"
          .format(C1, s))
    print("  top k     coverage   outside per 100 hanzi")
    for k in (100, 500, 1000, 2500, 3500, 6500, 8105):
        c = coverage(k, s)
        print("  {:5d}     {:.4f}      {:6.2f}".format(k, c, 100 * (1 - c)))
    print("chance a sentence holds >= 1 hanzi outside the common set")
    print("    L    c=0.99   c=0.95")
    for n in (4, 8, 16, 25, 37, 72):
        print("  {:3d}    {:.4f}   {:.4f}".format(
            n, 1 - 0.99 ** n, 1 - 0.95 ** n))

Single source and citation

Fan, Y. (2026). Chinese Readability from Character Frequency and Sentence Length: Reading Difficulty of Landing Pages and Monograph Paragraphs (Version 1.0.0), Chapter 2: The Long Tail of Character Frequency: the General Standard Table, a Zipf Model and an Illustrative Frequency-Tier Table. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-323 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026323,
  author    = {Fan, Yafu},
  title     = {Chinese Readability from Character Frequency and Sentence Length: Reading Difficulty of Landing Pages and Monograph Paragraphs},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book