关于「庖丁与斐德若:依自然关节切分的两种古老主张」,《分块与混合配比》给出的定量结论是什么?

结论:第七章考辨《庄子·养生主》庖丁解牛与柏拉图《斐德若篇》依自然关节切分:顺着段落切零重叠零折断,块长 64、重叠一半的固定窗口仍折断 0.401 的答案,而关节识别出错会迅速吃掉这一优势。

数据口径:书中参数为示意;全部计算由书中程序生成;程序不读写任何文件,不连接任何线上知识库;本书是非个人化经营建议,任何线上知识库参数的改动须经离线评测并由主事者人工复核放行。 第五章扫描混合权重:编号型查询最优 0.0,改写型最优 1.0,示意基线 0.7 / 0.3 在全改写型时遗憾 0.098;按类型分流在判别出错率为 0.10 时留出召回 0.928,已低于调好的全局权重 0.938。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.401, 64

chunkermean-lenmax-lenindexbroken
joints (paragraph)64.11121.000.000
fixed w= 64 o=0.0064.0641.000.633
fixed w= 64 o=0.2564.0641.330.515
fixed w= 64 o=0.5064.0642.000.401
fixed w=112 o=0.00112.01121.000.341
fixed w=112 o=0.50112.01122.000.018
查看完整验算程序
# Chapter 7. p71: cutting at the joints versus hacking at fixed windows.
# Model: a stream of 20000 paragraphs, lengths uniform on 16..112 tokens
# (seed 20261113); each paragraph is one answer. A paragraph is broken
# when no single chunk holds at least 80 % of it. Joint chunking cuts
# exactly at paragraph ends; fixed windows ignore them. Illustrative.
import math
import random

rng = random.Random(20261113)
lens = [rng.randint(16, 112) for _ in range(20000)]
starts, pos = [], 0
for ln in lens:
    starts.append(pos)
    pos += ln
total = pos


def broken(w, s):
    bad = 0
    for a, ln in zip(starts, lens):
        j0 = max(0, (a - w) // s)
        best = 0
        for j in range(j0, a // s + 2):
            lo = j * s
            best = max(best, min(lo + w, a + ln) - max(lo, a))
        bad += best < 0.8 * ln
    return bad / len(lens)


print("chunker            mean-len  max-len  index  broken")
print("joints (paragraph) %8.1f  %7d  %5.2f  %6.3f" % (
    math.fsum(lens) / len(lens), max(lens), 1.0, 0.0))
for w, o in ((64, 0.0), (64, 0.25), (64, 0.5), (112, 0.0), (112, 0.5)):
    s = int(round(w * (1 - o)))
    print("fixed w=%3d o=%.2f %8.1f  %7d  %5.2f  %6.3f" % (
        w, o, w, w, w / s, broken(w, s)))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《知识库分块边界网格寻优、重叠率与向量词频混合检索配比母典》第 7 章「庖丁与斐德若:依自然关节切分的两种古老主张」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-261 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026261,
  author    = {Fan, Yafu},
  title     = {Grid Search over Knowledge-Base Chunk Boundaries: Overlap Ratios and the Hybrid Weighting of Vector and Term-Frequency Retrieval},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论