关于「文字的指纹:瓦片、杰卡德系数与最小哈希」,《公域反垃圾检测与合规桥接页》给出的定量结论是什么?

结论:第三章用瓦片与最小哈希说明模板帖子两两相似度约为 0.522,只发 2 条就有 0.915309 的机会被分带检测抓到。

数据口径:书中参数为示意;全部计算由书中程序生成;本书是非个人化的经营分析,不构成经营、法律或理财建议,法规与平台规则以上架前人工复核为准。 书中以标注框区分已发表的证据、可运行的 Python 计算(20 段程序)、原典与作者的假说。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.522

wtemplate pairs: min Jmean J | original pairs: meanmax
10.8330.842 |0.060 0.188
20.6920.699 |0.002 0.026
30.5170.522 |0.000 0.000
40.3750.381 |0.000 0.000
50.2730.277 |0.000 0.000
w 3: original vs template mean J 0.002 over 64 pairs
w 3: containment of post 1 in each other template post 0.682
w 3: shingles per post, template 22 and original 18.9
查看完整验算程序
# Chapter 3. p31: word shingles, resemblance and containment.
# Model: Broder (1997) turns a text into its set S of w-word shingles
# (contiguous word runs) and measures resemblance r = |S1 & S2| / |S1 | S2|
# (the Jaccard coefficient) and containment c = |S1 & S2| / |S1|.
# Corpus (synthetic): 8 posts filled from one template with a product
# slot and 8 posts written separately on the same products. Shingles
# are hashed to 64-bit integers with hashlib so that p32 can sketch
# them. We report the pairwise Jaccard for shingle widths w = 1 to 5.
import hashlib
import itertools
import math
import statistics

ITEMS = ["desk lamp", "air fryer", "phone stand", "rice cooker",
         "yoga mat", "kettle", "travel bag", "monitor arm"]
TEMPLATE = ("big price drop on the {} today, tap my link to grab the {}"
            " deal before stock runs out and share with friends")
ORIGINAL = [
    "we measured the desk lamp at three heights and the glare on a matte"
    " screen only stayed low at the middle setting",
    "after a month the air fryer basket coating shows wear where the"
    " shaker hits it so check the warranty terms first",
    "the phone stand folds flat but the hinge loosens if you adjust it"
    " daily so tighten the screw every few weeks",
    "rice cooker test: brown rice needed an extra quarter cup of water"
    " and the keep warm mode dried the bottom layer",
    "a thicker yoga mat helped on tile floors but slid on wood, a cotton"
    " towel underneath fixed the slipping",
    "we timed the kettle from cold tap water and it boiled one litre in"
    " just under four minutes at full power",
    "the travel bag fits under most airline seats we tried, though the"
    " side pocket is too shallow for a large bottle",
    "a monitor arm needs a desk edge at least two centimetres thick, our"
    " thinner desk flexed when we moved the screen",
]


def shingles(text, w=3):
    words = text.lower().replace(",", " ").replace(":", " ").split()
    out = set()
    for i in range(len(words) - w + 1):
        g = " ".join(words[i:i + w]).encode("ascii")
        out.add(int.from_bytes(hashlib.md5(g).digest()[:8], "big"))
    return out


def jac(a, b):
    return len(a & b) / len(a | b)


def corpus():
    return [TEMPLATE.format(x, x) for x in ITEMS], ORIGINAL


def pairs_j(sets):
    return [jac(a, b) for a, b in itertools.combinations(sets, 2)]


if __name__ == "__main__":
    tpl, org = corpus()
    print("w   template pairs: min J  mean J | original pairs: mean  max")
    for w in (1, 2, 3, 4, 5):
        T = [shingles(t, w) for t in tpl]
        O = [shingles(t, w) for t in org]
        jt, jo = pairs_j(T), pairs_j(O)
        print("{}  {:>21.3f} {:>7.3f} | {:>21.3f} {:>5.3f}".format(
            w, min(jt), statistics.fmean(jt), statistics.fmean(jo),
            max(jo)))
    T = [shingles(t) for t in tpl]
    O = [shingles(t) for t in org]
    cross = [jac(a, b) for a in O for b in T]
    cont = [len(T[0] & t) / len(T[0]) for t in T[1:]]
    print("w 3: original vs template mean J {:.3f} over {} pairs".format(
        statistics.fmean(cross), len(cross)))
    print("w 3: containment of post 1 in each other template post {:.3f}"
          .format(math.fsum(cont) / len(cont)))
    print("w 3: shingles per post, template {} and original {:.1f}"
          .format(len(T[0]), statistics.fmean(len(o) for o in O)))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《公域社交图谱反垃圾检测边界、合规联盟桥接页转化与限流自愈动力学母典》第 3 章「文字的指纹:瓦片、杰卡德系数与最小哈希」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-342 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026342,
  author    = {Fan, Yafu},
  title     = {Social-Graph Anti-Spam Detection Bounds, Compliant Affiliate Bridge-Page Funnels and the Kinetics of Recovery after Reach Throttling},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论