关于「样本到来之后:插值偏差、第二矩的陷阱与换手点」,《最大熵需求重建》给出的定量结论是什么?

结论:先看均方根误差。样本只有 5 笔时,最大熵的误差是 45.2 元,样本分位数 52.4 元,正态 50.0 元;10 笔时分别是 34.1、44.4 与 38.0 元;20 笔时是 24.8、29.8 与 28.5 元。在二十笔以内,最大熵始终最准。原因很直接:样本分位数只用到排在最上面的一两笔订单,噪声极大;最大熵只用样本均值,均值是所有订单的平均,噪声小得多,再加上上下限这条免费的信息。

数据口径:本书推翻这一常识:正态捷径私加了对称形状与标准差,示意参数下均值 80、下限 30 时它把 0.133 的概率放在不可能的区间,均值 40 时升到 0.412;均匀捷径隐含均值在区间中点,直接违背已知均值。 只承认上下限与均值时,杰恩斯 1957 年提出的最大熵原则给出截断指数分布,示意甜品店的第 90 百分位客单为 146.0 元。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 10, 20, 24.8, 28.5, 29.8, 34.1

true P90 = 156.5; 2000 samples per n
n | max-ent biasrmse | sample biasrmse | normal biasrmse
5 |-10.845.2 |-35.552.4 |-25.250.0
10 |-11.934.1 |-25.944.4 |-24.038.0
20 |-10.324.8 |-12.929.8 |-20.828.5
50 |-10.117.6 |-4.816.4 |-20.223.5
200 |-10.412.7 |-0.97.1 |-20.020.9
查看完整验算程序
# Chapter 5. p51: rebuilding P90 from a small sample of real orders.
# Model: the true order values follow the illustrative two-hump mixture
# of p32 (on [30, 300], mean 80). A new shop sees n orders. Three ways
# to state P90: the maximum-entropy answer from the sample mean and the
# known bounds, the sample's own 90th percentile, and normal(xbar, s).
# 2,000 repetitions per n with a fixed seed; bias and root mean square
# error against the true P90 (bisection on a Simpson-rule CDF).
import math
import random
import statistics
from statistics import NormalDist

A, B, M = 30.0, 300.0, 80.0
L = B - A
FR = (M - A) / L


def simpson(f, lo, hi, n=2000):
    h = (hi - lo) / n
    terms = [f(lo), f(hi)]
    terms += [4.0 * f(lo + i * h) for i in range(1, n, 2)]
    terms += [2.0 * f(lo + i * h) for i in range(2, n, 2)]
    return math.fsum(terms) * h / 3.0


def integ(f, n=2000, hi=B):
    # x = A + v * v removes the kink of densities that start at A
    g = lambda v: f(A + v * v) * 2.0 * v
    return simpson(g, 0.0, math.sqrt(hi - A), n)


def te_mean(lam):
    u = lam * L
    if abs(u) < 1e-8:
        return A + L / 2.0
    return A + 1.0 / lam - L / math.expm1(u)


def solve(m):
    lo, hi = -2.0, 2.0
    for _ in range(80):
        mid = (lo + hi) / 2.0
        if te_mean(mid) > m:
            lo = mid
        else:
            hi = mid
    return (lo + hi) / 2.0


def te_pdf(lam):
    return lambda x: lam * math.exp(-lam * (x - A)) / -math.expm1(-lam * L)


def te_q(lam, p):
    return A - math.log1p(p * math.expm1(-lam * L)) / lam


def beta_pdf(u, al, be):
    if u <= 0.0 or u >= 1.0:
        return 0.0
    lg = math.lgamma(al + be) - math.lgamma(al) - math.lgamma(be)
    return math.exp(lg + (al - 1) * math.log(u) + (be - 1) * math.log1p(-u))


def beta_x(al, be):
    return lambda x: beta_pdf((x - A) / L, al, be) / L


F2 = (FR - 0.75 * 0.1) / 0.25
MIX = ((0.75, 4.0, 36.0), (0.25, 8.0, 8.0 * (1 - F2) / F2))


def mix_pdf(x):
    return math.fsum(w * beta_x(a, b)(x) for w, a, b in MIX)


def truths():
    lam = solve(M)
    return [("max-ent", te_pdf(lam)),
            ("b(2,8.8)", beta_x(2.0, 8.8)),
            ("b(1.2,5.28)", beta_x(1.2, 5.28)),
            ("two-hump", mix_pdf)]


def draw(rng):
    r = rng.random()
    w, a, b = MIX[0] if r < MIX[0][0] else MIX[1]
    return A + L * rng.betavariate(a, b)


def true_p90():
    lo, hi = A, B
    for _ in range(40):
        mid = (lo + hi) / 2.0
        if integ(mix_pdf, 400, mid) < 0.9:
            lo = mid
        else:
            hi = mid
    return (lo + hi) / 2.0


def stats(est, truth):
    bias = math.fsum(e - truth for e in est) / len(est)
    rmse = math.sqrt(math.fsum((e - truth) ** 2 for e in est) / len(est))
    return bias, rmse


if __name__ == "__main__":
    rng = random.Random(20261113)
    t90 = true_p90()
    print("true P90 = {:.1f}; 2000 samples per n".format(t90))
    print("     n | max-ent bias  rmse | sample bias  rmse |"
          " normal bias  rmse")
    for n in (5, 10, 20, 50, 200):
        me, em, no = [], [], []
        for _ in range(2000):
            xs = [draw(rng) for _ in range(n)]
            xbar = math.fsum(xs) / n
            me.append(te_q(solve(xbar), 0.9))
            em.append(statistics.quantiles(xs, n=10,
                                           method="inclusive")[8])
            no.append(NormalDist(xbar, statistics.stdev(xs)).inv_cdf(0.9))
        row = stats(me, t90) + stats(em, t90) + stats(no, t90)
        print("  {:>4} | {:>12.1f} {:>5.1f} | {:>11.1f} {:>5.1f} |"
              " {:>11.1f} {:>5.1f}".format(n, *row))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《最大熵原理与缺数据需求重建、仅知均价与上下限时最不偏的客单分布推断母典》第 5 章「样本到来之后:插值偏差、第二矩的陷阱与换手点」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-301 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026301,
  author    = {Fan, Yafu},
  title     = {The Maximum-Entropy Principle and Demand Reconstruction from Missing Data: the Least-Biased Order-Value Distribution Given Only a Mean and Bounds},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论