What does Maximum-Entropy Demand Reconstruction conclude, in numbers, about “After the Sample Arrives: Plug-in Bias, the Second-Moment Trap and the Handover Point”?

Answer: Look first at the root mean square error. With only 5 orders, maximum entropy errs by 45.2, the sample quantile by 52.4 and the normal by 50.0. With 10 orders the figures are 34.1, 44.4 and 38.0; with 20 they are 24.8, 29.8 and 28.5. Up to twenty orders maximum entropy is always the most accurate. The reason is direct. The sample quantile uses only the top one or two orders and is very noisy; maximum entropy uses only the sample mean, which averages over every order and is far less noisy, and it adds the free information of the bounds.

Scope of the numbers: The normal shortcut adds a symmetric shape and a standard deviation; under illustrative parameters, with a mean of 80 and a floor of 30, it puts 0.133 of its probability on impossible values, rising to 0.412 when the mean is 40. Accepting only the bounds and the mean, the maximum-entropy principle proposed by Jaynes in 1957 gives a truncated exponential distribution, with a 90th-percentile order of 146.0 for the illustrative dessert shop.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 10, 20, 24.8, 28.5, 29.8, 34.1

true P90 = 156.5; 2000 samples per n
n | max-ent biasrmse | sample biasrmse | normal biasrmse
5 |-10.845.2 |-35.552.4 |-25.250.0
10 |-11.934.1 |-25.944.4 |-24.038.0
20 |-10.324.8 |-12.929.8 |-20.828.5
50 |-10.117.6 |-4.816.4 |-20.223.5
200 |-10.412.7 |-0.97.1 |-20.020.9
Show the full program
# Chapter 5. p51: rebuilding P90 from a small sample of real orders.
# Model: the true order values follow the illustrative two-hump mixture
# of p32 (on [30, 300], mean 80). A new shop sees n orders. Three ways
# to state P90: the maximum-entropy answer from the sample mean and the
# known bounds, the sample's own 90th percentile, and normal(xbar, s).
# 2,000 repetitions per n with a fixed seed; bias and root mean square
# error against the true P90 (bisection on a Simpson-rule CDF).
import math
import random
import statistics
from statistics import NormalDist

A, B, M = 30.0, 300.0, 80.0
L = B - A
FR = (M - A) / L


def simpson(f, lo, hi, n=2000):
    h = (hi - lo) / n
    terms = [f(lo), f(hi)]
    terms += [4.0 * f(lo + i * h) for i in range(1, n, 2)]
    terms += [2.0 * f(lo + i * h) for i in range(2, n, 2)]
    return math.fsum(terms) * h / 3.0


def integ(f, n=2000, hi=B):
    # x = A + v * v removes the kink of densities that start at A
    g = lambda v: f(A + v * v) * 2.0 * v
    return simpson(g, 0.0, math.sqrt(hi - A), n)


def te_mean(lam):
    u = lam * L
    if abs(u) < 1e-8:
        return A + L / 2.0
    return A + 1.0 / lam - L / math.expm1(u)


def solve(m):
    lo, hi = -2.0, 2.0
    for _ in range(80):
        mid = (lo + hi) / 2.0
        if te_mean(mid) > m:
            lo = mid
        else:
            hi = mid
    return (lo + hi) / 2.0


def te_pdf(lam):
    return lambda x: lam * math.exp(-lam * (x - A)) / -math.expm1(-lam * L)


def te_q(lam, p):
    return A - math.log1p(p * math.expm1(-lam * L)) / lam


def beta_pdf(u, al, be):
    if u <= 0.0 or u >= 1.0:
        return 0.0
    lg = math.lgamma(al + be) - math.lgamma(al) - math.lgamma(be)
    return math.exp(lg + (al - 1) * math.log(u) + (be - 1) * math.log1p(-u))


def beta_x(al, be):
    return lambda x: beta_pdf((x - A) / L, al, be) / L


F2 = (FR - 0.75 * 0.1) / 0.25
MIX = ((0.75, 4.0, 36.0), (0.25, 8.0, 8.0 * (1 - F2) / F2))


def mix_pdf(x):
    return math.fsum(w * beta_x(a, b)(x) for w, a, b in MIX)


def truths():
    lam = solve(M)
    return [("max-ent", te_pdf(lam)),
            ("b(2,8.8)", beta_x(2.0, 8.8)),
            ("b(1.2,5.28)", beta_x(1.2, 5.28)),
            ("two-hump", mix_pdf)]


def draw(rng):
    r = rng.random()
    w, a, b = MIX[0] if r < MIX[0][0] else MIX[1]
    return A + L * rng.betavariate(a, b)


def true_p90():
    lo, hi = A, B
    for _ in range(40):
        mid = (lo + hi) / 2.0
        if integ(mix_pdf, 400, mid) < 0.9:
            lo = mid
        else:
            hi = mid
    return (lo + hi) / 2.0


def stats(est, truth):
    bias = math.fsum(e - truth for e in est) / len(est)
    rmse = math.sqrt(math.fsum((e - truth) ** 2 for e in est) / len(est))
    return bias, rmse


if __name__ == "__main__":
    rng = random.Random(20261113)
    t90 = true_p90()
    print("true P90 = {:.1f}; 2000 samples per n".format(t90))
    print("     n | max-ent bias  rmse | sample bias  rmse |"
          " normal bias  rmse")
    for n in (5, 10, 20, 50, 200):
        me, em, no = [], [], []
        for _ in range(2000):
            xs = [draw(rng) for _ in range(n)]
            xbar = math.fsum(xs) / n
            me.append(te_q(solve(xbar), 0.9))
            em.append(statistics.quantiles(xs, n=10,
                                           method="inclusive")[8])
            no.append(NormalDist(xbar, statistics.stdev(xs)).inv_cdf(0.9))
        row = stats(me, t90) + stats(em, t90) + stats(no, t90)
        print("  {:>4} | {:>12.1f} {:>5.1f} | {:>11.1f} {:>5.1f} |"
              " {:>11.1f} {:>5.1f}".format(n, *row))

Single source and citation

Fan, Y. (2026). The Maximum-Entropy Principle and Demand Reconstruction from Missing Data: the Least-Biased Order-Value Distribution Given Only a Mean and Bounds (Version 1.0.0), Chapter 5: After the Sample Arrives: Plug-in Bias, the Second-Moment Trap and the Handover Point. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-301 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026301,
  author    = {Fan, Yafu},
  title     = {The Maximum-Entropy Principle and Demand Reconstruction from Missing Data: the Least-Biased Order-Value Distribution Given Only a Mean and Bounds},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book