关于「段落长度与截断位置:切块、截断与内点最优」,《生成式搜索引用权重》给出的定量结论是什么?

结论:第三章讲切块与截断:截断在 128 个词时,段首结论被读到的比例是段尾的 4.5000 倍,段落长度的内点最优约为 60 到 130 个词。

数据口径:书中参数均为示意;全部计算由书中程序生成。 第五章讲可核查性,并用模拟说明从外部推断特征权重的脆弱:漏掉长度时统计数字的权重被高估 0.3252。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 128, 4.5000

share of paragraphs whose conclusion survives a cut at T
T first middle last
641.0000 0.1667 0.0556
1281.0000 0.5000 0.2222
2561.0000 1.0000 0.5556
5121.0000 1.0000 1.0000
mean paragraph length 230.0 tokens
at T128 conclusion-first keeps 1.0000, last keeps 0.2222
ratio first / last at T128: 4.5000
查看完整验算程序
# Chapter 3. p32: conclusion first versus conclusion last under a cut.
# A reranker or generator often reads only the first T tokens of each
# retrieved passage. A paragraph of n sentences of 20 tokens each puts
# its conclusion sentence first, in the middle, or last. Paragraph
# lengths run evenly from 3 to 20 sentences (60 to 400 tokens). The
# conclusion counts as read if its last token falls within T.
# Sentence length, cut points and length range are illustrative.
from fractions import Fraction as F

SENT = 20
LENGTHS = range(3, 21)


def end_token(n, where):
    if where == "first":
        return SENT
    if where == "middle":
        return (n // 2 + 1) * SENT
    return n * SENT


print("share of paragraphs whose conclusion survives a cut at T")
print("%6s %8s %8s %8s" % ("T", "first", "middle", "last"))
rows = {}
for t in (64, 128, 256, 512):
    vals = []
    for where in ("first", "middle", "last"):
        ok = len([n for n in LENGTHS if end_token(n, where) <= t])
        vals.append(F(ok, len(LENGTHS)))
    rows[t] = vals
    print("%6d %8.4f %8.4f %8.4f" % (t, float(vals[0]), float(vals[1]),
                                     float(vals[2])))
mean_len = F(SENT * (3 + 20), 2)
print("mean paragraph length %.1f tokens" % float(mean_len))
r = rows[128]
print("at T = 128 conclusion-first keeps %.4f, last keeps %.4f" % (
    float(r[0]), float(r[2])))
print("ratio first / last at T = 128: %.4f" % float(r[0] / r[2]))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《生成式搜索引用权重几何学、问答引擎检索重排特征与高被引双语直答页段落拓扑母典》第 3 章「段落长度与截断位置:切块、截断与内点最优」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-260 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026260,
  author    = {Fan, Yafu},
  title     = {The Geometry of Citation Weight in Generative Search: Retrieval and Reranking Features and the Paragraph Topology of Highly Cited Bilingual Answer Pages},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论