What does Pareto Whales and VIP Tiers conclude, in numbers, about “Power Law or Lognormal: Test the Tail Before Talking About Whales”?

Answer: Program p12 tries 60 candidate thresholds between the 0.50 and 0.98 quantiles and draws 200 synthetic books for each real book. The result is not what the long-tail chart would lead anyone to expect. For book LN the chosen threshold is 134.4, with 427 customers in the tail, an exponent of 2.574 and a distance of 0.0300. Its p-value is 0.430, comfortably above the bar, so the power law is not rejected. For book PT the chosen threshold is 125.4, not far from the true 120, with 329 customers in the tail and an exponent of 2.659, close to the true value of 1 plus 1.6. Yet its p-value is only 0.100, sitting exactly on the bar.

Scope of the numbers: Parameters are illustrative and all calculations are generated by the book's programs. This is general business analysis, not personalised legal, tax or financial advice, and it promises no income to anyone.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.0300, 0.100, 0.430, 125.4, 134.4, 2.574

bookx_minn_tailalphaKS DGOF p (200 reps)
LN134.44272.5740.03000.430
PT125.43292.6590.04530.100
rule: p < 0.1 rejects the power law (Clauset et al. 2009)
Show the full program
# Chapter 1. p12: Clauset-Shalizi-Newman fit with a bootstrap p-value.
# Model: for each candidate x_min (60 quantiles from 0.50 to 0.98) fit
# the continuous power law alpha = 1 + n / sum ln(x / x_min) and keep
# the x_min with the smallest Kolmogorov-Smirnov distance D. The
# goodness-of-fit p is the share of 200 semi-parametric bootstrap
# books whose own best D is at least the observed D (Clauset et al.
# 2009). Books LN and PT are the synthetic books of p11.
import math
import random


def make_books(n):
    rng = random.Random(39901)
    ln = [math.exp(rng.gauss(3.6, 1.25)) for _ in range(n)]
    pt = []
    for _ in range(n):
        if rng.random() < 0.12:
            pt.append(120.0 * (1.0 - rng.random()) ** (-1.0 / 1.6))
        else:
            pt.append(math.exp(rng.gauss(3.0, 0.6)))
    return ln, pt


def fit_tail(s, start):
    tail = s[start:]
    m = len(tail)
    xm = tail[0]
    a = 1.0 + m / math.fsum(math.log(v / xm) for v in tail)
    d = 0.0
    for i, v in enumerate(tail):
        model = 1.0 - (v / xm) ** (1.0 - a)
        d = max(d, abs((i + 1) / m - model), abs(i / m - model))
    return xm, a, d, m


def best_fit(xs):
    s = sorted(xs)
    n = len(s)
    best = None
    for j in range(60):
        q = 0.50 + 0.48 * j / 59
        r = fit_tail(s, int(q * n))
        if best is None or r[2] < best[2]:
            best = r
    return best


def gof_p(xs, fit, reps, rng):
    xm, a, d0, m = fit
    body = [v for v in xs if v < xm]
    n = len(xs)
    hits = 0
    for _ in range(reps):
        syn = []
        for _ in range(n):
            if rng.random() < m / n or not body:
                u = 1.0 - rng.random()
                syn.append(xm * u ** (-1.0 / (a - 1.0)))
            else:
                syn.append(body[rng.randrange(len(body))])
        if best_fit(syn)[2] >= d0:
            hits += 1
    return hits / reps


if __name__ == "__main__":
    ln, pt = make_books(3000)
    rng = random.Random(39902)
    print("book   x_min    n_tail  alpha   KS D     GOF p (200 reps)")
    for name, xs in (("LN", ln), ("PT", pt)):
        f = best_fit(xs)
        p = gof_p(xs, f, 200, rng)
        print("{:<5} {:>7.1f}  {:>6,}  {:>5.3f}  {:>7.4f}  {:>6.3f}".format(
            name, f[0], f[3], f[1], f[2], p))
    print("rule: p < 0.1 rejects the power law (Clauset et al. 2009)")

Single source and citation

Fan, Y. (2026). Pareto Whale Auto-Detection, High-Net-Worth Self-Selection Signals and Ten-X VIP Tier Escalation (Version 1.0.0), Chapter 1: Power Law or Lognormal: Test the Tail Before Talking About Whales. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-399 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026399,
  author    = {Fan, Yafu},
  title     = {Pareto Whale Auto-Detection, High-Net-Worth Self-Selection Signals and Ten-X VIP Tier Escalation},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book