关于「ROC 与阈值:把预检做成可校准的判别」,《品牌撞名三维预检》给出的定量结论是什么?

结论:再看两种合成方式。它们的 AUC 几乎一样,自助区间从 -0.016 到 0.005,包含零。换句话说,就整体排序而言,二者没有可分辨的差别。差别出现在固定阈值上。阈值都取 0.6 时,加权平均的查全率只有 0.295,取最大值是 0.797;代价是误报率,前者 0.021,后者 0.268。阈值都取 0.8 时,加权平均只抓住 0.152,取最大值抓住 0.531。

数据口径:在一个示意模型中,声音通道占八成时音码的 AUC 为 0.851,高于字母编辑距离的 0.817。 书中参数为示意;全部计算由书中程序生成;本书是非个人化法律建议,商标近似判断以各地官方检索与专业代理意见为准。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: -0.016, 0.005, 0.021, 0.152, 0.268, 0.295

pairs 4000 confusions 684
AUC sound0.776
AUC shape0.709
AUC meaning0.621
AUC mean(0.4,0.3,0.3)0.833
AUC max0.828
cut mean: TPR FPR max: TPR FPR
0.50.472 0.076 0.893 0.407
0.60.295 0.021 0.797 0.268
0.70.240 0.008 0.675 0.182
0.80.152 0.003 0.531 0.120
AUC(max) - AUC(mean)95% bootstrap interval -0.016 to 0.005
查看完整验算程序
# Chapter 6. p61: ROC of single axes, the weighted mean and the max.
# Model: 4000 synthetic name pairs (fixed seed). Kind of pair: none
# near 0.70, sound-alike 0.10, shape-alike 0.10, meaning-alike 0.05,
# near on all three 0.05. A near axis draws beta(8, 2), a far axis
# beta(2, 5). A confusion happens through hearing (0.50), reading
# (0.35) or recalling a meaning (0.15), with logit P = -6 + 9 * the
# similarity on that channel. AUC by mid-ranks; the 95% interval for
# AUC(max) - AUC(mean) uses 300 bootstrap resamples. All illustrative.
import math
import random

KINDS = ((0.70, (0, 0, 0)), (0.10, (1, 0, 0)), (0.10, (0, 1, 0)),
         (0.05, (0, 0, 1)), (0.05, (1, 1, 1)))
CHANNEL = (0.50, 0.35, 0.15)
W = (0.4, 0.3, 0.3)


def auc(scores, labels):
    allv = sorted(zip(scores, labels))
    ranks, i = [], 0
    while i < len(allv):
        j = i
        while j < len(allv) and allv[j][0] == allv[i][0]:
            j += 1
        mid = (i + 1 + j) / 2.0
        ranks.extend(mid for k in range(i, j) if allv[k][1])
        i = j
    n1 = len(ranks)
    n0 = len(allv) - n1
    return (math.fsum(ranks) - n1 * (n1 + 1) / 2.0) / (n1 * n0)


def pick(rng, probs):
    u, acc = rng.random(), 0.0
    for k, p in enumerate(probs):
        acc = math.fsum([acc, p])
        if u < acc:
            return k
    return len(probs) - 1


def simulate(rng, n):
    rows = []
    for _ in range(n):
        near = KINDS[pick(rng, [p for p, _ in KINDS])][1]
        s = tuple(rng.betavariate(8, 2) if f else rng.betavariate(2, 5)
                  for f in near)
        ch = pick(rng, CHANNEL)
        p = 1.0 / (1.0 + math.exp(6.0 - 9.0 * s[ch]))
        mean = math.fsum(w * x for w, x in zip(W, s))
        rows.append((s[0], s[1], s[2], mean, max(s), rng.random() < p))
    return rows


def rates(rows, col, cut):
    pos = [r for r in rows if r[5]]
    neg = [r for r in rows if not r[5]]
    tp = len([r for r in pos if r[col] >= cut])
    fp = len([r for r in neg if r[col] >= cut])
    return tp / len(pos), fp / len(neg)


if __name__ == "__main__":
    rng = random.Random(20261113)
    rows = simulate(rng, 4000)
    lab = [r[5] for r in rows]
    print("pairs {}   confusions {}".format(len(rows), lab.count(True)))
    names = ("sound", "shape", "meaning", "mean(0.4,0.3,0.3)", "max")
    for c, nm in enumerate(names):
        print("AUC {:18s} {:.3f}".format(nm, auc([r[c] for r in rows],
                                                 lab)))
    print("cut   mean: TPR   FPR    max: TPR   FPR")
    for cut in (0.5, 0.6, 0.7, 0.8):
        t1, f1 = rates(rows, 3, cut)
        t2, f2 = rates(rows, 4, cut)
        print("{:.1f}  {:10.3f} {:5.3f} {:10.3f} {:5.3f}".format(
            cut, t1, f1, t2, f2))
    diffs = []
    for _ in range(300):
        bs = [rows[rng.randrange(len(rows))] for _ in rows]
        bl = [r[5] for r in bs]
        diffs.append(auc([r[4] for r in bs], bl) -
                     auc([r[3] for r in bs], bl))
    diffs.sort()
    print("AUC(max) - AUC(mean): 95% bootstrap interval {:.3f} to {:.3f}"
          .format(diffs[7], diffs[292]))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《商标音近形近义近三维距离、从反切到音码的品牌撞名预检母典》第 6 章「ROC 与阈值:把预检做成可校准的判别」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-317 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026317,
  author    = {Fan, Yafu},
  title     = {Phonetic, Graphic and Semantic Distance between Trademarks: Pre-Screening Brand-Name Collisions from Fanqie Spellings to Sound Codes},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论