关于「重复计数:相关特征让朴素贝叶斯过度自信」,《奢品多特征似然比初筛》给出的定量结论是什么?

结论:第二章是全书核心:示意参数下,朴素后验判到 0.99 以上、平均 0.996 的货,实际不符只有 0.756;放大倍数随类内相关升高,在相关系数 0.6 时达到 2.28;但排序并不吃亏,伤害集中在远离先验的边界上。

数据口径:书中全部概率只是外观概率参考,不是官方鉴定证书,不构成真伪结论,涉及争议或高价值货品须送品牌或专业鉴定机构;品牌一律以通称指代;书中参数为示意,全部计算由书中程序生成;本书是非个人化经营建议。 第二章是全书核心:示意参数下,朴素后验判到 0.99 以上、平均 0.996 的货,实际不符只有 0.756;放大倍数随类内相关升高,在相关系数 0.6 时达到 2.28;但排序并不吃亏,伤害集中在远离先验的边界上。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.756, 0.99, 0.996

rho = 0.5, 40000 items, prior P(N) = 0.10
P(N) binnaive: nmeanobs N |exact: nmeanobs N
0.00-0.01196130.0020.015 |86940.0050.004
0.01-0.0564330.0250.055 |142290.0260.026
0.05-0.2050460.1060.105 |110700.1030.106
0.20-0.5034160.3290.173 |43290.3140.323
0.50-0.8023120.6500.285 |13330.6330.614
0.80-0.9516130.8830.395 |3130.8670.856
0.95-0.998710.9740.557 |290.9660.966
0.99-1.006960.9960.756 |30.9931.000
naive: Brier 0.0947log loss 0.3396share beyond 0.01/0.99 0.508
exact: Brier 0.0671log loss 0.2298share beyond 0.01/0.99 0.217
查看完整验算程序
# Chapter 2. p21: naive Bayes counts correlated evidence twice. Four
# feature scores with an equal within-class correlation rho = 0.5 (a
# shared factor such as wear, age, lighting or the same operator), the
# same separations d as p12, prior P(N) = 0.10, 40,000 simulated items
# (fixed seed). Reliability table: predicted P(N) against the observed
# share of N, for the naive posterior and for the exact posterior that
# uses the correlation. All parameters are illustrative.
import math
import random

D = [1.2, 1.0, 1.5, 0.9]
RHO = 0.5
PRIOR_N = 0.10
NSIM = 40000
EDGES = [0.0, 0.01, 0.05, 0.2, 0.5, 0.8, 0.95, 0.99, 1.0000001]


def rinv_equi(n, rho):
    # 等相关矩阵的逆:(I - c J) / (1 - rho),c = rho / (1 + (n - 1) rho)
    c = rho / (1.0 + (n - 1) * rho)
    return [[((1.0 if i == j else 0.0) - c) / (1.0 - rho)
             for j in range(n)] for i in range(n)]


def matvec(a, x):
    return [math.fsum(r * v for r, v in zip(row, x)) for row in a]


def dot(x, y):
    return math.fsum(a * b for a, b in zip(x, y))


RI = rinv_equi(4, RHO)
W = matvec(RI, D)            # exact weights R^-1 d
C0 = dot(D, W) / 2.0
N0 = dot(D, D) / 2.0


def sigmoid(w, prior):
    o = math.log(prior / (1.0 - prior)) + w
    if o > 0:
        return 1.0 / (1.0 + math.exp(-o))
    e = math.exp(o)
    return e / (1.0 + e)


rng = random.Random(20261113)
rows = []
for _ in range(NSIM):
    y = 1 if rng.random() < PRIOR_N else 0
    f = rng.gauss(0.0, 1.0)
    z = [y * d + math.sqrt(RHO) * f + math.sqrt(1.0 - RHO)
         * rng.gauss(0.0, 1.0) for d in D]
    naive = dot(D, z) - N0
    exact = dot(W, z) - C0
    rows.append((y, sigmoid(naive, PRIOR_N), sigmoid(exact, PRIOR_N)))


def table(k):
    out = []
    for lo, hi in zip(EDGES[:-1], EDGES[1:]):
        sel = [r for r in rows if lo <= r[k] < hi]
        if sel:
            out.append((len(sel), math.fsum(r[k] for r in sel) / len(sel),
                        len([r for r in sel if r[0] == 1]) / len(sel)))
        else:
            out.append((0, 0.0, 0.0))
    return out


tn, te = table(1), table(2)
print("rho = %.1f, %d items, prior P(N) = %.2f" % (RHO, NSIM, PRIOR_N))
print("P(N) bin        naive: n   mean   obs N |  exact: n   mean   obs N")
for (lo, hi), a, b in zip(zip(EDGES[:-1], EDGES[1:]), tn, te):
    lab = "%.2f-%.2f" % (lo, min(hi, 1.0))
    print("%-12s %9d %6.3f %7.3f | %8d %6.3f %7.3f"
          % (lab, a[0], a[1], a[2], b[0], b[1], b[2]))
for k, name in ((1, "naive"), (2, "exact")):
    br = math.fsum((r[k] - r[0]) ** 2 for r in rows) / NSIM
    ll = -math.fsum(math.log(max(r[k] if r[0] else 1.0 - r[k], 1e-300))
                    for r in rows) / NSIM
    ext = len([r for r in rows if r[k] < 0.01 or r[k] > 0.99]) / NSIM
    print("%s: Brier %.4f  log loss %.4f  share beyond 0.01/0.99 %.3f"
          % (name, br, ll, ext))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《奢品真伪多特征似然比鉴定、走线节距五金光谱皮质纹理融合与送检秒收贝叶斯边界母典》第 2 章「重复计数:相关特征让朴素贝叶斯过度自信」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-281 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026281,
  author    = {Fan, Yafu},
  title     = {Multi-Feature Likelihood-Ratio Screening of Pre-Owned Luxury Goods: Fusing Stitch Pitch, Hardware Spectra and Leather Grain, and the Bayesian Boundary between Lab Referral and Instant Purchase},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论