What does Three-Way Brand Collision Pre-Screening conclude, in numbers, about “ROC Curves and Thresholds: Making the Pre-Screen a Calibrated Classifier”?
Answer: Now consider the two ways of combining. Their AUCs are almost equal; the bootstrap interval runs from -0.016 to 0.005 and contains zero. As a ranking, the two cannot be told apart. The difference appears at fixed thresholds. With a threshold of 0.6 for both, the weighted mean has a recall of only 0.295 against 0.797 for the maximum; the price is the false-alarm rate, 0.021 against 0.268. With a threshold of 0.8 for both, the weighted mean catches only 0.152 of the confusions and the maximum 0.531.
Scope of the numbers: In an illustrative model, with the sound channel at eight tenths, the AUC of the sound code is 0.851 against 0.817 for letter edit distance. Parameters are illustrative; all calculations are generated by the book's programs; the book is general analysis, not individualised legal advice, and judgements of trademark similarity rest with official searches in each jurisdiction and professional agents.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: -0.016, 0.005, 0.021, 0.152, 0.268, 0.295
| pairs 4000 confusions 684 | |
| AUC sound | 0.776 |
| AUC shape | 0.709 |
| AUC meaning | 0.621 |
| AUC mean(0.4,0.3,0.3) | 0.833 |
| AUC max | 0.828 |
| cut mean: TPR FPR max: TPR FPR | |
| 0.5 | 0.472 0.076 0.893 0.407 |
| 0.6 | 0.295 0.021 0.797 0.268 |
| 0.7 | 0.240 0.008 0.675 0.182 |
| 0.8 | 0.152 0.003 0.531 0.120 |
| AUC(max) - AUC(mean) | 95% bootstrap interval -0.016 to 0.005 |
Show the full program
# Chapter 6. p61: ROC of single axes, the weighted mean and the max.
# Model: 4000 synthetic name pairs (fixed seed). Kind of pair: none
# near 0.70, sound-alike 0.10, shape-alike 0.10, meaning-alike 0.05,
# near on all three 0.05. A near axis draws beta(8, 2), a far axis
# beta(2, 5). A confusion happens through hearing (0.50), reading
# (0.35) or recalling a meaning (0.15), with logit P = -6 + 9 * the
# similarity on that channel. AUC by mid-ranks; the 95% interval for
# AUC(max) - AUC(mean) uses 300 bootstrap resamples. All illustrative.
import math
import random
KINDS = ((0.70, (0, 0, 0)), (0.10, (1, 0, 0)), (0.10, (0, 1, 0)),
(0.05, (0, 0, 1)), (0.05, (1, 1, 1)))
CHANNEL = (0.50, 0.35, 0.15)
W = (0.4, 0.3, 0.3)
def auc(scores, labels):
allv = sorted(zip(scores, labels))
ranks, i = [], 0
while i < len(allv):
j = i
while j < len(allv) and allv[j][0] == allv[i][0]:
j += 1
mid = (i + 1 + j) / 2.0
ranks.extend(mid for k in range(i, j) if allv[k][1])
i = j
n1 = len(ranks)
n0 = len(allv) - n1
return (math.fsum(ranks) - n1 * (n1 + 1) / 2.0) / (n1 * n0)
def pick(rng, probs):
u, acc = rng.random(), 0.0
for k, p in enumerate(probs):
acc = math.fsum([acc, p])
if u < acc:
return k
return len(probs) - 1
def simulate(rng, n):
rows = []
for _ in range(n):
near = KINDS[pick(rng, [p for p, _ in KINDS])][1]
s = tuple(rng.betavariate(8, 2) if f else rng.betavariate(2, 5)
for f in near)
ch = pick(rng, CHANNEL)
p = 1.0 / (1.0 + math.exp(6.0 - 9.0 * s[ch]))
mean = math.fsum(w * x for w, x in zip(W, s))
rows.append((s[0], s[1], s[2], mean, max(s), rng.random() < p))
return rows
def rates(rows, col, cut):
pos = [r for r in rows if r[5]]
neg = [r for r in rows if not r[5]]
tp = len([r for r in pos if r[col] >= cut])
fp = len([r for r in neg if r[col] >= cut])
return tp / len(pos), fp / len(neg)
if __name__ == "__main__":
rng = random.Random(20261113)
rows = simulate(rng, 4000)
lab = [r[5] for r in rows]
print("pairs {} confusions {}".format(len(rows), lab.count(True)))
names = ("sound", "shape", "meaning", "mean(0.4,0.3,0.3)", "max")
for c, nm in enumerate(names):
print("AUC {:18s} {:.3f}".format(nm, auc([r[c] for r in rows],
lab)))
print("cut mean: TPR FPR max: TPR FPR")
for cut in (0.5, 0.6, 0.7, 0.8):
t1, f1 = rates(rows, 3, cut)
t2, f2 = rates(rows, 4, cut)
print("{:.1f} {:10.3f} {:5.3f} {:10.3f} {:5.3f}".format(
cut, t1, f1, t2, f2))
diffs = []
for _ in range(300):
bs = [rows[rng.randrange(len(rows))] for _ in rows]
bl = [r[5] for r in bs]
diffs.append(auc([r[4] for r in bs], bl) -
auc([r[3] for r in bs], bl))
diffs.sort()
print("AUC(max) - AUC(mean): 95% bootstrap interval {:.3f} to {:.3f}"
.format(diffs[7], diffs[292]))Single source and citation
Fan, Y. (2026). Phonetic, Graphic and Semantic Distance between Trademarks: Pre-Screening Brand-Name Collisions from Fanqie Spellings to Sound Codes (Version 1.0.0), Chapter 6: ROC Curves and Thresholds: Making the Pre-Screen a Calibrated Classifier. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-317 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026317,
author = {Fan, Yafu},
title = {Phonetic, Graphic and Semantic Distance between Trademarks: Pre-Screening Brand-Name Collisions from Fanqie Spellings to Sound Codes},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Far in Spelling, Near in Sound: Edit Distance, Sound Skeletons and the Channels of Confusion
- Chapter 2 · Soundex: a Century-Old Sound Code, Too Wide and Too Narrow
- Chapter 3 · Initial, Final and Tone: a Distance for Chinese Syllables and Homophones across Languages
- Chapter 4 · Component Distance: a Table of Look-Alike Characters and Unicode Normalisation
- Chapter 5 · Meaning and a Three-Way Score: How a Weighted Mean Dilutes a One-Dimension Collision
- Chapter 7 · East and West in the Sources: the Preface to the Qieyun and Russell's Patent of 1918
- Chapter 8 · A Pre-Screening Loop for New Brand Names, and a Plan to Refute the Book