关于「段落长度与截断位置:切块、截断与内点最优」,《生成式搜索引用权重》给出的定量结论是什么?
结论:第三章讲切块与截断:截断在 128 个词时,段首结论被读到的比例是段尾的 4.5000 倍,段落长度的内点最优约为 60 到 130 个词。
数据口径:书中参数均为示意;全部计算由书中程序生成。 第五章讲可核查性,并用模拟说明从外部推断特征权重的脆弱:漏掉长度时统计数字的权重被高估 0.3252。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 128, 4.5000
| share of paragraphs whose conclusion survives a cut at T | |
| T first middle last | |
| 64 | 1.0000 0.1667 0.0556 |
| 128 | 1.0000 0.5000 0.2222 |
| 256 | 1.0000 1.0000 0.5556 |
| 512 | 1.0000 1.0000 1.0000 |
| mean paragraph length 230.0 tokens | |
| at T | 128 conclusion-first keeps 1.0000, last keeps 0.2222 |
| ratio first / last at T | 128: 4.5000 |
查看完整验算程序
# Chapter 3. p32: conclusion first versus conclusion last under a cut.
# A reranker or generator often reads only the first T tokens of each
# retrieved passage. A paragraph of n sentences of 20 tokens each puts
# its conclusion sentence first, in the middle, or last. Paragraph
# lengths run evenly from 3 to 20 sentences (60 to 400 tokens). The
# conclusion counts as read if its last token falls within T.
# Sentence length, cut points and length range are illustrative.
from fractions import Fraction as F
SENT = 20
LENGTHS = range(3, 21)
def end_token(n, where):
if where == "first":
return SENT
if where == "middle":
return (n // 2 + 1) * SENT
return n * SENT
print("share of paragraphs whose conclusion survives a cut at T")
print("%6s %8s %8s %8s" % ("T", "first", "middle", "last"))
rows = {}
for t in (64, 128, 256, 512):
vals = []
for where in ("first", "middle", "last"):
ok = len([n for n in LENGTHS if end_token(n, where) <= t])
vals.append(F(ok, len(LENGTHS)))
rows[t] = vals
print("%6d %8.4f %8.4f %8.4f" % (t, float(vals[0]), float(vals[1]),
float(vals[2])))
mean_len = F(SENT * (3 + 20), 2)
print("mean paragraph length %.1f tokens" % float(mean_len))
r = rows[128]
print("at T = 128 conclusion-first keeps %.4f, last keeps %.4f" % (
float(r[0]), float(r[2])))
print("ratio first / last at T = 128: %.4f" % float(r[0] / r[2]))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《生成式搜索引用权重几何学、问答引擎检索重排特征与高被引双语直答页段落拓扑母典》第 3 章「段落长度与截断位置:切块、截断与内点最优」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-260 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026260,
author = {Fan, Yafu},
title = {The Geometry of Citation Weight in Generative Search: Retrieval and Reranking Features and the Paragraph Topology of Highly Cited Bilingual Answer Pages},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}