关于「文字的指纹:瓦片、杰卡德系数与最小哈希」,《公域反垃圾检测与合规桥接页》给出的定量结论是什么?
结论:第三章用瓦片与最小哈希说明模板帖子两两相似度约为 0.522,只发 2 条就有 0.915309 的机会被分带检测抓到。
数据口径:书中参数为示意;全部计算由书中程序生成;本书是非个人化的经营分析,不构成经营、法律或理财建议,法规与平台规则以上架前人工复核为准。 书中以标注框区分已发表的证据、可运行的 Python 计算(20 段程序)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.522
| w | template pairs: min J | mean J | original pairs: mean | max |
|---|---|---|---|
| 1 | 0.833 | 0.842 | | 0.060 0.188 |
| 2 | 0.692 | 0.699 | | 0.002 0.026 |
| 3 | 0.517 | 0.522 | | 0.000 0.000 |
| 4 | 0.375 | 0.381 | | 0.000 0.000 |
| 5 | 0.273 | 0.277 | | 0.000 0.000 |
| w 3: original vs template mean J 0.002 over 64 pairs | |||
| w 3: containment of post 1 in each other template post 0.682 | |||
| w 3: shingles per post, template 22 and original 18.9 |
查看完整验算程序
# Chapter 3. p31: word shingles, resemblance and containment.
# Model: Broder (1997) turns a text into its set S of w-word shingles
# (contiguous word runs) and measures resemblance r = |S1 & S2| / |S1 | S2|
# (the Jaccard coefficient) and containment c = |S1 & S2| / |S1|.
# Corpus (synthetic): 8 posts filled from one template with a product
# slot and 8 posts written separately on the same products. Shingles
# are hashed to 64-bit integers with hashlib so that p32 can sketch
# them. We report the pairwise Jaccard for shingle widths w = 1 to 5.
import hashlib
import itertools
import math
import statistics
ITEMS = ["desk lamp", "air fryer", "phone stand", "rice cooker",
"yoga mat", "kettle", "travel bag", "monitor arm"]
TEMPLATE = ("big price drop on the {} today, tap my link to grab the {}"
" deal before stock runs out and share with friends")
ORIGINAL = [
"we measured the desk lamp at three heights and the glare on a matte"
" screen only stayed low at the middle setting",
"after a month the air fryer basket coating shows wear where the"
" shaker hits it so check the warranty terms first",
"the phone stand folds flat but the hinge loosens if you adjust it"
" daily so tighten the screw every few weeks",
"rice cooker test: brown rice needed an extra quarter cup of water"
" and the keep warm mode dried the bottom layer",
"a thicker yoga mat helped on tile floors but slid on wood, a cotton"
" towel underneath fixed the slipping",
"we timed the kettle from cold tap water and it boiled one litre in"
" just under four minutes at full power",
"the travel bag fits under most airline seats we tried, though the"
" side pocket is too shallow for a large bottle",
"a monitor arm needs a desk edge at least two centimetres thick, our"
" thinner desk flexed when we moved the screen",
]
def shingles(text, w=3):
words = text.lower().replace(",", " ").replace(":", " ").split()
out = set()
for i in range(len(words) - w + 1):
g = " ".join(words[i:i + w]).encode("ascii")
out.add(int.from_bytes(hashlib.md5(g).digest()[:8], "big"))
return out
def jac(a, b):
return len(a & b) / len(a | b)
def corpus():
return [TEMPLATE.format(x, x) for x in ITEMS], ORIGINAL
def pairs_j(sets):
return [jac(a, b) for a, b in itertools.combinations(sets, 2)]
if __name__ == "__main__":
tpl, org = corpus()
print("w template pairs: min J mean J | original pairs: mean max")
for w in (1, 2, 3, 4, 5):
T = [shingles(t, w) for t in tpl]
O = [shingles(t, w) for t in org]
jt, jo = pairs_j(T), pairs_j(O)
print("{} {:>21.3f} {:>7.3f} | {:>21.3f} {:>5.3f}".format(
w, min(jt), statistics.fmean(jt), statistics.fmean(jo),
max(jo)))
T = [shingles(t) for t in tpl]
O = [shingles(t) for t in org]
cross = [jac(a, b) for a in O for b in T]
cont = [len(T[0] & t) / len(T[0]) for t in T[1:]]
print("w 3: original vs template mean J {:.3f} over {} pairs".format(
statistics.fmean(cross), len(cross)))
print("w 3: containment of post 1 in each other template post {:.3f}"
.format(math.fsum(cont) / len(cont)))
print("w 3: shingles per post, template {} and original {:.1f}"
.format(len(T[0]), statistics.fmean(len(o) for o in O)))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《公域社交图谱反垃圾检测边界、合规联盟桥接页转化与限流自愈动力学母典》第 3 章「文字的指纹:瓦片、杰卡德系数与最小哈希」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-342 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026342,
author = {Fan, Yafu},
title = {Social-Graph Anti-Spam Detection Bounds, Compliant Affiliate Bridge-Page Funnels and the Kinetics of Recovery after Reach Throttling},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}