What does Extreme Tails and Chargeback Reserves conclude, in numbers, about “The Mean Excess Plot and the Choice of Threshold: Bias, Variance and the Lognormal Trap”?

Answer: Chapter 5 chooses the threshold with the mean excess plot and 150 simulated shops; the log error of the ten-year level is smallest at the ninetieth percentile, at 0.194.

Scope of the numbers: Chapter 5 chooses the threshold with the mean excess plot and 150 simulated shops; the log error of the ten-year level is smallest at the ninetieth percentile, at 0.194. Marked boxes separate published evidence, runnable Python calculations (19 programs), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.194, 150

exact 120-month level 27214 yuan; 150 shops x 43200 orders
qxi medxi 5%-95%sigma*level ratiorms log
0.800.3620.329-0.38956.00.8300.241
0.900.3830.334-0.41939.20.9310.194
0.950.3870.330-0.44733.50.9550.217
0.980.3890.309-0.48124.00.9650.270
0.990.3910.283-0.51623.70.9380.308
Show the full program
# Chapter 5. p52: where to put the threshold - bias against variance.
# Model: 150 simulated shops, each with 24 months of 1,800 orders from
# the illustrative Burr XII law (b = 200, c = 2, k = 1.25), fixed seed.
# For thresholds at the empirical 0.80 to 0.99 quantiles we fit a GPD
# by PWM, then report: the median and 90% range of xi, the modified
# scale sigma* = sigma - xi u (flat when the GPD holds), and the error
# of the 120-month return level against its exact value, as a median
# ratio and a root mean square of the log ratio.

import math
import random

B, C, K = 200.0, 2.0, 1.25
PER, MONTHS, REPS = 1800, 24, 150


def burr_q(p):
    return B * ((1.0 - p) ** (-1.0 / K) - 1.0) ** (1.0 / C)


def gpd_pwm(y):
    ys = sorted(y)
    n = len(ys)
    a0 = math.fsum(ys) / n
    a1 = math.fsum(v * (n - 1 - j) / (n - 1) for j, v in enumerate(ys)) / n
    xi = 2.0 - a0 / (a0 - 2.0 * a1)
    return xi, 2.0 * a0 * a1 / (a0 - 2.0 * a1)


def pct(xs, p):
    ys = sorted(xs)
    return ys[min(len(ys) - 1, int(p * len(ys)))]


QS = (0.80, 0.90, 0.95, 0.98, 0.99)
exact = burr_q(1.0 - 1.0 / (PER * 120))
rng = random.Random(20261122)
xi_s = {q: [] for q in QS}
ss_s = {q: [] for q in QS}
lr_s = {q: [] for q in QS}
for _ in range(REPS):
    xs = sorted(burr_q(rng.random()) for _ in range(PER * MONTHS))
    n = len(xs)
    for q in QS:
        u = xs[int(q * n)]
        y = [x - u for x in xs if x > u]
        xi, sig = gpd_pwm(y)
        lev = u + sig / xi * ((len(y) / MONTHS * 120) ** xi - 1.0)
        xi_s[q].append(xi)
        ss_s[q].append(sig - xi * u)
        lr_s[q].append(math.log(lev / exact))
print("exact 120-month level %.0f yuan; %d shops x %d orders"
      % (exact, REPS, PER * MONTHS))
print("    q    xi med   xi 5%-95%     sigma*  level ratio  rms log")
for q in QS:
    lr = lr_s[q]
    rms = math.sqrt(math.fsum(v * v for v in lr) / REPS)
    print("  %4.2f  %6.3f  %5.3f-%5.3f  %8.1f  %10.3f  %7.3f"
          % (q, pct(xi_s[q], 0.5), pct(xi_s[q], 0.05), pct(xi_s[q], 0.95),
             pct(ss_s[q], 0.5), math.exp(pct(lr, 0.5)), rms))

Single source and citation

Fan, Y. (2026). Extreme-Value Theory and Generalised Pareto Tails: Whale-Order Ceilings and Reserves for Extreme Chargebacks (Version 1.0.0), Chapter 5: The Mean Excess Plot and the Choice of Threshold: Bias, Variance and the Lognormal Trap. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-244 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026244,
  author    = {Fan, Yafu},
  title     = {Extreme-Value Theory and Generalised Pareto Tails: Whale-Order Ceilings and Reserves for Extreme Chargebacks},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book