What does Maximum-Entropy Demand Reconstruction conclude, in numbers, about “Adding the Second Moment: the Truncated Normal Family, Convex Exponents and What One Standard Deviation Buys”?

Answer: At standard deviations of 60 and 75 the maximum-entropy answer becomes convex. The 90th percentile is pushed to 166.1 and 236.8 while the median drops to 57.0 and 47.9: polarisation empties the middle. The uncut normal breaks down completely here. Its 10th percentiles are 3.1 and \(-16.1\), its out-of-range shares 0.202 and 0.252, and its 90th percentiles 156.9 and 176.1, both far below the maximum-entropy answer. At a standard deviation of 75 entropy falls by 0.497 bits, the largest drop in the table, because a standard deviation far above the implied value is a very surprising piece of information.

Scope of the numbers: The normal shortcut adds a symmetric shape and a standard deviation; under illustrative parameters, with a mean of 80 and a floor of 30, it puts 0.133 of its probability on impossible values, rising to 0.412 when the mean is 40. Accepting only the bounds and the mean, the maximum-entropy principle proposed by Jaynes in 1957 gives a truncated exponential distribution, with a 90th-percentile order of 146.0 for the illustrative dessert shop.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: -16.1, 0.202, 0.252, 0.497, 10, 156.9

mean only: P10 35.4 P50 65.4 P90 146.0 H 7.080
sd max-ent P10 P50 P90 bits gained | normal P10 P90 <30
3042.477.0 121.00.235 |41.6 118.4 0.048
4037.170.7 136.30.036 |28.7 131.3 0.106
6033.757.0 166.10.094 |3.1 156.9 0.202
7532.447.9 236.80.497 |-16.1 176.1 0.252
Show the full program
# Chapter 4. p42: what a stated spread buys, in bits and in quantiles.
# Model: as p41 (bounds [30, 300], mean 80, illustrative). For each
# stated standard deviation we print the two-moment maximum-entropy
# quantiles, the entropy it gives up against the mean-only answer
# (the information the spread adds), and the quantiles an uncut
# normal(80, sd) would claim, with the share it puts below 30.
import math
from statistics import NormalDist

A, B, M = 30.0, 300.0, 80.0
L = B - A
NODES = 2000


def grid():
    h = 1.0 / NODES
    us = [i * h for i in range(NODES + 1)]
    cs = [1.0 if i in (0, NODES) else (4.0 if i % 2 else 2.0)
          for i in range(NODES + 1)]
    return us, cs


US, CS = grid()
UP = [[u ** k for u in US] for k in range(1, 5)]


def moments(l1, l2, kmax=4):
    # Simpson weights times exp(-(l1 u + l2 u^2)) on u in [0, 1]
    ex = [-(l1 * u + l2 * u * u) for u in US]
    top = max(ex)
    w = [c * math.exp(e - top) for c, e in zip(CS, ex)]
    z = math.fsum(w)
    mom = [math.fsum(wi * uk for wi, uk in zip(w, UP[k - 1])) / z
           for k in range(1, kmax + 1)]
    logz = math.log(z / (3.0 * NODES)) + top
    return mom, logz


def solve2(mean, sd, l1=0.0, l2=0.0):
    # Newton on (l1, l2) so that E[x] = mean and Var[x] = sd^2
    t1 = (mean - A) / L
    t2 = t1 * t1 + (sd / L) ** 2
    for _ in range(60):
        (m1, m2, m3, m4), _ = moments(l1, l2)
        g1, g2 = m1 - t1, m2 - t2
        if abs(g1) + abs(g2) < 1e-13:
            break
        c11 = m2 - m1 * m1
        c12 = m3 - m1 * m2
        c22 = m4 - m2 * m2
        det = c11 * c22 - c12 * c12
        d1 = (c22 * g1 - c12 * g2) / det
        d2 = (c11 * g2 - c12 * g1) / det
        step = 1.0
        while step > 1e-4:
            n1, n2 = l1 + step * d1, l2 + step * d2
            (k1, k2, _, _), _ = moments(n1, n2)
            if abs(k1 - t1) + abs(k2 - t2) < abs(g1) + abs(g2):
                break
            step /= 2.0
        l1, l2 = n1, n2
    return l1, l2


def entropy_bits(l1, l2):
    (m1, m2, _, _), logz = moments(l1, l2)
    return (logz + l1 * m1 + l2 * m2 + math.log(L)) / math.log(2.0)


def part(l1, l2, hi, n=400):
    # integral of exp(-(l1 u + l2 u^2)) over [0, hi], Simpson rule
    h = hi / n
    f = lambda u: math.exp(-(l1 * u + l2 * u * u))
    terms = [f(0.0), f(hi)]
    terms += [4.0 * f(i * h) for i in range(1, n, 2)]
    terms += [2.0 * f(i * h) for i in range(2, n, 2)]
    return math.fsum(terms) * h / 3.0


def quantile2(l1, l2, p):
    z = part(l1, l2, 1.0, NODES)
    lo, hi = 0.0, 1.0
    for _ in range(40):
        mid = (lo + hi) / 2.0
        if part(l1, l2, mid) / z < p:
            lo = mid
        else:
            hi = mid
    return A + L * (lo + hi) / 2.0


def solve1():
    lo, hi = -50.0, 50.0
    for _ in range(80):
        mid = (lo + hi) / 2.0
        (m1, _, _, _), _ = moments(mid, 0.0)
        lo, hi = (mid, hi) if m1 > (M - A) / L else (lo, mid)
    return (lo + hi) / 2.0


if __name__ == "__main__":
    l0 = solve1()
    h0 = entropy_bits(l0, 0.0)
    q0 = [quantile2(l0, 0.0, p) for p in (0.1, 0.5, 0.9)]
    print("mean only:  P10 {:5.1f}  P50 {:5.1f}  P90 {:5.1f}  H {:.3f}"
          .format(q0[0], q0[1], q0[2], h0))
    print("    sd  max-ent P10  P50   P90   bits gained |"
          " normal P10  P90  <30")
    for sd in (30.0, 40.0, 60.0, 75.0):
        l1, l2 = solve2(M, sd)
        q = [quantile2(l1, l2, p) for p in (0.1, 0.5, 0.9)]
        nd = NormalDist(M, sd)
        print("  {:>4.0f}  {:>11.1f} {:>5.1f} {:>5.1f}   {:>11.3f} |"
              " {:>10.1f} {:>5.1f} {:.3f}".format(
                  sd, q[0], q[1], q[2], h0 - entropy_bits(l1, l2),
                  nd.inv_cdf(0.1), nd.inv_cdf(0.9), nd.cdf(A)))

Single source and citation

Fan, Y. (2026). The Maximum-Entropy Principle and Demand Reconstruction from Missing Data: the Least-Biased Order-Value Distribution Given Only a Mean and Bounds (Version 1.0.0), Chapter 4: Adding the Second Moment: the Truncated Normal Family, Convex Exponents and What One Standard Deviation Buys. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-301 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026301,
  author    = {Fan, Yafu},
  title     = {The Maximum-Entropy Principle and Demand Reconstruction from Missing Data: the Least-Biased Order-Value Distribution Given Only a Mean and Bounds},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book