关于「庖丁与斐德若:依自然关节切分的两种古老主张」,《分块与混合配比》给出的定量结论是什么?
结论:第七章考辨《庄子·养生主》庖丁解牛与柏拉图《斐德若篇》依自然关节切分:顺着段落切零重叠零折断,块长 64、重叠一半的固定窗口仍折断 0.401 的答案,而关节识别出错会迅速吃掉这一优势。
数据口径:书中参数为示意;全部计算由书中程序生成;程序不读写任何文件,不连接任何线上知识库;本书是非个人化经营建议,任何线上知识库参数的改动须经离线评测并由主事者人工复核放行。 第五章扫描混合权重:编号型查询最优 0.0,改写型最优 1.0,示意基线 0.7 / 0.3 在全改写型时遗憾 0.098;按类型分流在判别出错率为 0.10 时留出召回 0.928,已低于调好的全局权重 0.938。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 0.401, 64
| chunker | mean-len | max-len | index | broken |
|---|---|---|---|---|
| joints (paragraph) | 64.1 | 112 | 1.00 | 0.000 |
| fixed w= 64 o=0.00 | 64.0 | 64 | 1.00 | 0.633 |
| fixed w= 64 o=0.25 | 64.0 | 64 | 1.33 | 0.515 |
| fixed w= 64 o=0.50 | 64.0 | 64 | 2.00 | 0.401 |
| fixed w=112 o=0.00 | 112.0 | 112 | 1.00 | 0.341 |
| fixed w=112 o=0.50 | 112.0 | 112 | 2.00 | 0.018 |
查看完整验算程序
# Chapter 7. p71: cutting at the joints versus hacking at fixed windows.
# Model: a stream of 20000 paragraphs, lengths uniform on 16..112 tokens
# (seed 20261113); each paragraph is one answer. A paragraph is broken
# when no single chunk holds at least 80 % of it. Joint chunking cuts
# exactly at paragraph ends; fixed windows ignore them. Illustrative.
import math
import random
rng = random.Random(20261113)
lens = [rng.randint(16, 112) for _ in range(20000)]
starts, pos = [], 0
for ln in lens:
starts.append(pos)
pos += ln
total = pos
def broken(w, s):
bad = 0
for a, ln in zip(starts, lens):
j0 = max(0, (a - w) // s)
best = 0
for j in range(j0, a // s + 2):
lo = j * s
best = max(best, min(lo + w, a + ln) - max(lo, a))
bad += best < 0.8 * ln
return bad / len(lens)
print("chunker mean-len max-len index broken")
print("joints (paragraph) %8.1f %7d %5.2f %6.3f" % (
math.fsum(lens) / len(lens), max(lens), 1.0, 0.0))
for w, o in ((64, 0.0), (64, 0.25), (64, 0.5), (112, 0.0), (112, 0.5)):
s = int(round(w * (1 - o)))
print("fixed w=%3d o=%.2f %8.1f %7d %5.2f %6.3f" % (
w, o, w, w, w / s, broken(w, s)))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《知识库分块边界网格寻优、重叠率与向量词频混合检索配比母典》第 7 章「庖丁与斐德若:依自然关节切分的两种古老主张」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-261 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026261,
author = {Fan, Yafu},
title = {Grid Search over Knowledge-Base Chunk Boundaries: Overlap Ratios and the Hybrid Weighting of Vector and Term-Frequency Retrieval},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}