关于「重复计数:相关特征让朴素贝叶斯过度自信」,《奢品多特征似然比初筛》给出的定量结论是什么?
结论:第二章是全书核心:示意参数下,朴素后验判到 0.99 以上、平均 0.996 的货,实际不符只有 0.756;放大倍数随类内相关升高,在相关系数 0.6 时达到 2.28;但排序并不吃亏,伤害集中在远离先验的边界上。
数据口径:书中全部概率只是外观概率参考,不是官方鉴定证书,不构成真伪结论,涉及争议或高价值货品须送品牌或专业鉴定机构;品牌一律以通称指代;书中参数为示意,全部计算由书中程序生成;本书是非个人化经营建议。 第二章是全书核心:示意参数下,朴素后验判到 0.99 以上、平均 0.996 的货,实际不符只有 0.756;放大倍数随类内相关升高,在相关系数 0.6 时达到 2.28;但排序并不吃亏,伤害集中在远离先验的边界上。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.756, 0.99, 0.996
| rho = 0.5, 40000 items, prior P(N) = 0.10 | ||||||
| P(N) bin | naive: n | mean | obs N | | exact: n | mean | obs N |
| 0.00-0.01 | 19613 | 0.002 | 0.015 | | 8694 | 0.005 | 0.004 |
| 0.01-0.05 | 6433 | 0.025 | 0.055 | | 14229 | 0.026 | 0.026 |
| 0.05-0.20 | 5046 | 0.106 | 0.105 | | 11070 | 0.103 | 0.106 |
| 0.20-0.50 | 3416 | 0.329 | 0.173 | | 4329 | 0.314 | 0.323 |
| 0.50-0.80 | 2312 | 0.650 | 0.285 | | 1333 | 0.633 | 0.614 |
| 0.80-0.95 | 1613 | 0.883 | 0.395 | | 313 | 0.867 | 0.856 |
| 0.95-0.99 | 871 | 0.974 | 0.557 | | 29 | 0.966 | 0.966 |
| 0.99-1.00 | 696 | 0.996 | 0.756 | | 3 | 0.993 | 1.000 |
| naive: Brier 0.0947 | log loss 0.3396 | share beyond 0.01/0.99 0.508 | ||||
| exact: Brier 0.0671 | log loss 0.2298 | share beyond 0.01/0.99 0.217 |
查看完整验算程序
# Chapter 2. p21: naive Bayes counts correlated evidence twice. Four
# feature scores with an equal within-class correlation rho = 0.5 (a
# shared factor such as wear, age, lighting or the same operator), the
# same separations d as p12, prior P(N) = 0.10, 40,000 simulated items
# (fixed seed). Reliability table: predicted P(N) against the observed
# share of N, for the naive posterior and for the exact posterior that
# uses the correlation. All parameters are illustrative.
import math
import random
D = [1.2, 1.0, 1.5, 0.9]
RHO = 0.5
PRIOR_N = 0.10
NSIM = 40000
EDGES = [0.0, 0.01, 0.05, 0.2, 0.5, 0.8, 0.95, 0.99, 1.0000001]
def rinv_equi(n, rho):
# 等相关矩阵的逆:(I - c J) / (1 - rho),c = rho / (1 + (n - 1) rho)
c = rho / (1.0 + (n - 1) * rho)
return [[((1.0 if i == j else 0.0) - c) / (1.0 - rho)
for j in range(n)] for i in range(n)]
def matvec(a, x):
return [math.fsum(r * v for r, v in zip(row, x)) for row in a]
def dot(x, y):
return math.fsum(a * b for a, b in zip(x, y))
RI = rinv_equi(4, RHO)
W = matvec(RI, D) # exact weights R^-1 d
C0 = dot(D, W) / 2.0
N0 = dot(D, D) / 2.0
def sigmoid(w, prior):
o = math.log(prior / (1.0 - prior)) + w
if o > 0:
return 1.0 / (1.0 + math.exp(-o))
e = math.exp(o)
return e / (1.0 + e)
rng = random.Random(20261113)
rows = []
for _ in range(NSIM):
y = 1 if rng.random() < PRIOR_N else 0
f = rng.gauss(0.0, 1.0)
z = [y * d + math.sqrt(RHO) * f + math.sqrt(1.0 - RHO)
* rng.gauss(0.0, 1.0) for d in D]
naive = dot(D, z) - N0
exact = dot(W, z) - C0
rows.append((y, sigmoid(naive, PRIOR_N), sigmoid(exact, PRIOR_N)))
def table(k):
out = []
for lo, hi in zip(EDGES[:-1], EDGES[1:]):
sel = [r for r in rows if lo <= r[k] < hi]
if sel:
out.append((len(sel), math.fsum(r[k] for r in sel) / len(sel),
len([r for r in sel if r[0] == 1]) / len(sel)))
else:
out.append((0, 0.0, 0.0))
return out
tn, te = table(1), table(2)
print("rho = %.1f, %d items, prior P(N) = %.2f" % (RHO, NSIM, PRIOR_N))
print("P(N) bin naive: n mean obs N | exact: n mean obs N")
for (lo, hi), a, b in zip(zip(EDGES[:-1], EDGES[1:]), tn, te):
lab = "%.2f-%.2f" % (lo, min(hi, 1.0))
print("%-12s %9d %6.3f %7.3f | %8d %6.3f %7.3f"
% (lab, a[0], a[1], a[2], b[0], b[1], b[2]))
for k, name in ((1, "naive"), (2, "exact")):
br = math.fsum((r[k] - r[0]) ** 2 for r in rows) / NSIM
ll = -math.fsum(math.log(max(r[k] if r[0] else 1.0 - r[k], 1e-300))
for r in rows) / NSIM
ext = len([r for r in rows if r[k] < 0.01 or r[k] > 0.99]) / NSIM
print("%s: Brier %.4f log loss %.4f share beyond 0.01/0.99 %.3f"
% (name, br, ll, ext))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《奢品真伪多特征似然比鉴定、走线节距五金光谱皮质纹理融合与送检秒收贝叶斯边界母典》第 2 章「重复计数:相关特征让朴素贝叶斯过度自信」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-281 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026281,
author = {Fan, Yafu},
title = {Multi-Feature Likelihood-Ratio Screening of Pre-Owned Luxury Goods: Fusing Stitch Pitch, Hardware Spectra and Leather Grain, and the Bayesian Boundary between Lab Referral and Instant Purchase},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}