What does Maximum-Entropy Demand Reconstruction conclude, in numbers, about “After the Sample Arrives: Plug-in Bias, the Second-Moment Trap and the Handover Point”?
Answer: Look first at the root mean square error. With only 5 orders, maximum entropy errs by 45.2, the sample quantile by 52.4 and the normal by 50.0. With 10 orders the figures are 34.1, 44.4 and 38.0; with 20 they are 24.8, 29.8 and 28.5. Up to twenty orders maximum entropy is always the most accurate. The reason is direct. The sample quantile uses only the top one or two orders and is very noisy; maximum entropy uses only the sample mean, which averages over every order and is far less noisy, and it adds the free information of the bounds.
Scope of the numbers: The normal shortcut adds a symmetric shape and a standard deviation; under illustrative parameters, with a mean of 80 and a floor of 30, it puts 0.133 of its probability on impossible values, rising to 0.412 when the mean is 40. Accepting only the bounds and the mean, the maximum-entropy principle proposed by Jaynes in 1957 gives a truncated exponential distribution, with a 90th-percentile order of 146.0 for the illustrative dessert shop.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 10, 20, 24.8, 28.5, 29.8, 34.1
| true P90 = 156.5; 2000 samples per n | ||||||
| n | max-ent bias | rmse | sample bias | rmse | normal bias | rmse | |||
| 5 | | -10.8 | 45.2 | | -35.5 | 52.4 | | -25.2 | 50.0 |
| 10 | | -11.9 | 34.1 | | -25.9 | 44.4 | | -24.0 | 38.0 |
| 20 | | -10.3 | 24.8 | | -12.9 | 29.8 | | -20.8 | 28.5 |
| 50 | | -10.1 | 17.6 | | -4.8 | 16.4 | | -20.2 | 23.5 |
| 200 | | -10.4 | 12.7 | | -0.9 | 7.1 | | -20.0 | 20.9 |
Show the full program
# Chapter 5. p51: rebuilding P90 from a small sample of real orders.
# Model: the true order values follow the illustrative two-hump mixture
# of p32 (on [30, 300], mean 80). A new shop sees n orders. Three ways
# to state P90: the maximum-entropy answer from the sample mean and the
# known bounds, the sample's own 90th percentile, and normal(xbar, s).
# 2,000 repetitions per n with a fixed seed; bias and root mean square
# error against the true P90 (bisection on a Simpson-rule CDF).
import math
import random
import statistics
from statistics import NormalDist
A, B, M = 30.0, 300.0, 80.0
L = B - A
FR = (M - A) / L
def simpson(f, lo, hi, n=2000):
h = (hi - lo) / n
terms = [f(lo), f(hi)]
terms += [4.0 * f(lo + i * h) for i in range(1, n, 2)]
terms += [2.0 * f(lo + i * h) for i in range(2, n, 2)]
return math.fsum(terms) * h / 3.0
def integ(f, n=2000, hi=B):
# x = A + v * v removes the kink of densities that start at A
g = lambda v: f(A + v * v) * 2.0 * v
return simpson(g, 0.0, math.sqrt(hi - A), n)
def te_mean(lam):
u = lam * L
if abs(u) < 1e-8:
return A + L / 2.0
return A + 1.0 / lam - L / math.expm1(u)
def solve(m):
lo, hi = -2.0, 2.0
for _ in range(80):
mid = (lo + hi) / 2.0
if te_mean(mid) > m:
lo = mid
else:
hi = mid
return (lo + hi) / 2.0
def te_pdf(lam):
return lambda x: lam * math.exp(-lam * (x - A)) / -math.expm1(-lam * L)
def te_q(lam, p):
return A - math.log1p(p * math.expm1(-lam * L)) / lam
def beta_pdf(u, al, be):
if u <= 0.0 or u >= 1.0:
return 0.0
lg = math.lgamma(al + be) - math.lgamma(al) - math.lgamma(be)
return math.exp(lg + (al - 1) * math.log(u) + (be - 1) * math.log1p(-u))
def beta_x(al, be):
return lambda x: beta_pdf((x - A) / L, al, be) / L
F2 = (FR - 0.75 * 0.1) / 0.25
MIX = ((0.75, 4.0, 36.0), (0.25, 8.0, 8.0 * (1 - F2) / F2))
def mix_pdf(x):
return math.fsum(w * beta_x(a, b)(x) for w, a, b in MIX)
def truths():
lam = solve(M)
return [("max-ent", te_pdf(lam)),
("b(2,8.8)", beta_x(2.0, 8.8)),
("b(1.2,5.28)", beta_x(1.2, 5.28)),
("two-hump", mix_pdf)]
def draw(rng):
r = rng.random()
w, a, b = MIX[0] if r < MIX[0][0] else MIX[1]
return A + L * rng.betavariate(a, b)
def true_p90():
lo, hi = A, B
for _ in range(40):
mid = (lo + hi) / 2.0
if integ(mix_pdf, 400, mid) < 0.9:
lo = mid
else:
hi = mid
return (lo + hi) / 2.0
def stats(est, truth):
bias = math.fsum(e - truth for e in est) / len(est)
rmse = math.sqrt(math.fsum((e - truth) ** 2 for e in est) / len(est))
return bias, rmse
if __name__ == "__main__":
rng = random.Random(20261113)
t90 = true_p90()
print("true P90 = {:.1f}; 2000 samples per n".format(t90))
print(" n | max-ent bias rmse | sample bias rmse |"
" normal bias rmse")
for n in (5, 10, 20, 50, 200):
me, em, no = [], [], []
for _ in range(2000):
xs = [draw(rng) for _ in range(n)]
xbar = math.fsum(xs) / n
me.append(te_q(solve(xbar), 0.9))
em.append(statistics.quantiles(xs, n=10,
method="inclusive")[8])
no.append(NormalDist(xbar, statistics.stdev(xs)).inv_cdf(0.9))
row = stats(me, t90) + stats(em, t90) + stats(no, t90)
print(" {:>4} | {:>12.1f} {:>5.1f} | {:>11.1f} {:>5.1f} |"
" {:>11.1f} {:>5.1f}".format(n, *row))Single source and citation
Fan, Y. (2026). The Maximum-Entropy Principle and Demand Reconstruction from Missing Data: the Least-Biased Order-Value Distribution Given Only a Mean and Bounds (Version 1.0.0), Chapter 5: After the Sample Arrives: Plug-in Bias, the Second-Moment Trap and the Handover Point. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-301 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026301,
author = {Fan, Yafu},
title = {The Maximum-Entropy Principle and Demand Reconstruction from Missing Data: the Least-Biased Order-Value Distribution Given Only a Mean and Bounds},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Lagrange Multiplier of Maximum Entropy: the Truncated Exponential When Only a Mean and Bounds Are Known
- Chapter 2 · Discrete Maximum Entropy over Price Tiers: Wallis's Count, the Concentration Theorem and the Limits of Uniqueness
- Chapter 3 · The Cost of the Normal and Uniform Shortcuts: Quantiles, Log Loss and the Minimax
- Chapter 4 · Adding the Second Moment: the Truncated Normal Family, Convex Exponents and What One Standard Deviation Buys
- Chapter 6 · Stocking a New Product with the Newsvendor Model: Ordering at a Maximum-Entropy Quantile and Worst-Case Regret
- Chapter 7 · East and West in the Classics: "Knowing That One Does Not Know" in Laozi 71 and Laplace's Philosophical Essay on Probabilities
- Chapter 8 · Closing the Loop: Cold-Start Demand for a New Product, the Maximum-Entropy Demand Reconstructor and a Pre-Registered Plan of Refutation