What does Maximum-Entropy Demand Reconstruction conclude, in numbers, about “Adding the Second Moment: the Truncated Normal Family, Convex Exponents and What One Standard Deviation Buys”?
Answer: At standard deviations of 60 and 75 the maximum-entropy answer becomes convex. The 90th percentile is pushed to 166.1 and 236.8 while the median drops to 57.0 and 47.9: polarisation empties the middle. The uncut normal breaks down completely here. Its 10th percentiles are 3.1 and \(-16.1\), its out-of-range shares 0.202 and 0.252, and its 90th percentiles 156.9 and 176.1, both far below the maximum-entropy answer. At a standard deviation of 75 entropy falls by 0.497 bits, the largest drop in the table, because a standard deviation far above the implied value is a very surprising piece of information.
Scope of the numbers: The normal shortcut adds a symmetric shape and a standard deviation; under illustrative parameters, with a mean of 80 and a floor of 30, it puts 0.133 of its probability on impossible values, rising to 0.412 when the mean is 40. Accepting only the bounds and the mean, the maximum-entropy principle proposed by Jaynes in 1957 gives a truncated exponential distribution, with a 90th-percentile order of 146.0 for the illustrative dessert shop.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: -16.1, 0.202, 0.252, 0.497, 10, 156.9
| mean only: P10 35.4 P50 65.4 P90 146.0 H 7.080 | ||||
| sd max-ent P10 P50 P90 bits gained | normal P10 P90 <30 | ||||
| 30 | 42.4 | 77.0 121.0 | 0.235 | | 41.6 118.4 0.048 |
| 40 | 37.1 | 70.7 136.3 | 0.036 | | 28.7 131.3 0.106 |
| 60 | 33.7 | 57.0 166.1 | 0.094 | | 3.1 156.9 0.202 |
| 75 | 32.4 | 47.9 236.8 | 0.497 | | -16.1 176.1 0.252 |
Show the full program
# Chapter 4. p42: what a stated spread buys, in bits and in quantiles.
# Model: as p41 (bounds [30, 300], mean 80, illustrative). For each
# stated standard deviation we print the two-moment maximum-entropy
# quantiles, the entropy it gives up against the mean-only answer
# (the information the spread adds), and the quantiles an uncut
# normal(80, sd) would claim, with the share it puts below 30.
import math
from statistics import NormalDist
A, B, M = 30.0, 300.0, 80.0
L = B - A
NODES = 2000
def grid():
h = 1.0 / NODES
us = [i * h for i in range(NODES + 1)]
cs = [1.0 if i in (0, NODES) else (4.0 if i % 2 else 2.0)
for i in range(NODES + 1)]
return us, cs
US, CS = grid()
UP = [[u ** k for u in US] for k in range(1, 5)]
def moments(l1, l2, kmax=4):
# Simpson weights times exp(-(l1 u + l2 u^2)) on u in [0, 1]
ex = [-(l1 * u + l2 * u * u) for u in US]
top = max(ex)
w = [c * math.exp(e - top) for c, e in zip(CS, ex)]
z = math.fsum(w)
mom = [math.fsum(wi * uk for wi, uk in zip(w, UP[k - 1])) / z
for k in range(1, kmax + 1)]
logz = math.log(z / (3.0 * NODES)) + top
return mom, logz
def solve2(mean, sd, l1=0.0, l2=0.0):
# Newton on (l1, l2) so that E[x] = mean and Var[x] = sd^2
t1 = (mean - A) / L
t2 = t1 * t1 + (sd / L) ** 2
for _ in range(60):
(m1, m2, m3, m4), _ = moments(l1, l2)
g1, g2 = m1 - t1, m2 - t2
if abs(g1) + abs(g2) < 1e-13:
break
c11 = m2 - m1 * m1
c12 = m3 - m1 * m2
c22 = m4 - m2 * m2
det = c11 * c22 - c12 * c12
d1 = (c22 * g1 - c12 * g2) / det
d2 = (c11 * g2 - c12 * g1) / det
step = 1.0
while step > 1e-4:
n1, n2 = l1 + step * d1, l2 + step * d2
(k1, k2, _, _), _ = moments(n1, n2)
if abs(k1 - t1) + abs(k2 - t2) < abs(g1) + abs(g2):
break
step /= 2.0
l1, l2 = n1, n2
return l1, l2
def entropy_bits(l1, l2):
(m1, m2, _, _), logz = moments(l1, l2)
return (logz + l1 * m1 + l2 * m2 + math.log(L)) / math.log(2.0)
def part(l1, l2, hi, n=400):
# integral of exp(-(l1 u + l2 u^2)) over [0, hi], Simpson rule
h = hi / n
f = lambda u: math.exp(-(l1 * u + l2 * u * u))
terms = [f(0.0), f(hi)]
terms += [4.0 * f(i * h) for i in range(1, n, 2)]
terms += [2.0 * f(i * h) for i in range(2, n, 2)]
return math.fsum(terms) * h / 3.0
def quantile2(l1, l2, p):
z = part(l1, l2, 1.0, NODES)
lo, hi = 0.0, 1.0
for _ in range(40):
mid = (lo + hi) / 2.0
if part(l1, l2, mid) / z < p:
lo = mid
else:
hi = mid
return A + L * (lo + hi) / 2.0
def solve1():
lo, hi = -50.0, 50.0
for _ in range(80):
mid = (lo + hi) / 2.0
(m1, _, _, _), _ = moments(mid, 0.0)
lo, hi = (mid, hi) if m1 > (M - A) / L else (lo, mid)
return (lo + hi) / 2.0
if __name__ == "__main__":
l0 = solve1()
h0 = entropy_bits(l0, 0.0)
q0 = [quantile2(l0, 0.0, p) for p in (0.1, 0.5, 0.9)]
print("mean only: P10 {:5.1f} P50 {:5.1f} P90 {:5.1f} H {:.3f}"
.format(q0[0], q0[1], q0[2], h0))
print(" sd max-ent P10 P50 P90 bits gained |"
" normal P10 P90 <30")
for sd in (30.0, 40.0, 60.0, 75.0):
l1, l2 = solve2(M, sd)
q = [quantile2(l1, l2, p) for p in (0.1, 0.5, 0.9)]
nd = NormalDist(M, sd)
print(" {:>4.0f} {:>11.1f} {:>5.1f} {:>5.1f} {:>11.3f} |"
" {:>10.1f} {:>5.1f} {:.3f}".format(
sd, q[0], q[1], q[2], h0 - entropy_bits(l1, l2),
nd.inv_cdf(0.1), nd.inv_cdf(0.9), nd.cdf(A)))Single source and citation
Fan, Y. (2026). The Maximum-Entropy Principle and Demand Reconstruction from Missing Data: the Least-Biased Order-Value Distribution Given Only a Mean and Bounds (Version 1.0.0), Chapter 4: Adding the Second Moment: the Truncated Normal Family, Convex Exponents and What One Standard Deviation Buys. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-301 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026301,
author = {Fan, Yafu},
title = {The Maximum-Entropy Principle and Demand Reconstruction from Missing Data: the Least-Biased Order-Value Distribution Given Only a Mean and Bounds},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Lagrange Multiplier of Maximum Entropy: the Truncated Exponential When Only a Mean and Bounds Are Known
- Chapter 2 · Discrete Maximum Entropy over Price Tiers: Wallis's Count, the Concentration Theorem and the Limits of Uniqueness
- Chapter 3 · The Cost of the Normal and Uniform Shortcuts: Quantiles, Log Loss and the Minimax
- Chapter 5 · After the Sample Arrives: Plug-in Bias, the Second-Moment Trap and the Handover Point
- Chapter 6 · Stocking a New Product with the Newsvendor Model: Ordering at a Maximum-Entropy Quantile and Worst-Case Regret
- Chapter 7 · East and West in the Classics: "Knowing That One Does Not Know" in Laozi 71 and Laplace's Philosophical Essay on Probabilities
- Chapter 8 · Closing the Loop: Cold-Start Demand for a New Product, the Maximum-Entropy Demand Reconstructor and a Pre-Registered Plan of Refutation