What does Citation Weight in Generative Search conclude, in numbers, about “The Geometry of Hybrid Scoring: BM25 Saturation, Cosine Direction and Rank Fusion”?
Answer: Chapter 2 covers BM25 saturation, cosine direction and rank fusion: one new true matching term earns nearly three times what one repeated term does.
Scope of the numbers: Its parameters are illustrative; all calculations are generated by its programs. It is not individual business advice.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python only
| k1 = 1.2, b = 0.75, avgdl = 60, idf = 2.0 | |||||
| extra | dl | w(t1) | w(t2) | total | marginal |
| 0 | 60 | 2.0000 | 2.0000 | 4.0000 | - |
| 1 | 61 | 2.7372 | 1.9865 | 4.7236 | +0.7236 |
| 2 | 62 | 3.1206 | 1.9731 | 5.0937 | +0.3700 |
| 3 | 63 | 3.3556 | 1.9599 | 5.3155 | +0.2218 |
| 5 | 65 | 3.6289 | 1.9341 | 5.5629 | +0.1237 |
| 8 | 68 | 3.8372 | 1.8966 | 5.7338 | +0.0569 |
| 12 | 72 | 3.9777 | 1.8487 | 5.8265 | +0.0232 |
| 20 | 80 | 4.1067 | 1.7600 | 5.8667 | +0.0050 |
| 40 | 100 | 4.2150 | 1.5714 | 5.7864 | -0.0040 |
| ceiling of w(t1) as tf grows: idf (k1 + 1) = 4.4000 | |||||
| one new matching term t3 : total 5.9594 (+1.9594) | |||||
| one more repetition of t1: total 4.7236 (+0.7236) |
Key formula
Show the full program
# Chapter 2. p21: BM25 saturation, the arithmetic behind stuffing.
# Okapi BM25 term weight (Robertson and Zaragoza 2009):
# w = idf * tf (k1 + 1) / (tf + k1 (1 - b + b dl / avgdl))
# A 60-token passage answers a two-term query (t1, t2) once each. We
# repeat t1 r more times (each repetition adds one token) and compare
# with adding one new matching term t3 of the same idf instead.
# k1 = 1.2, b = 0.75 are common defaults; idf = 2.0 is illustrative.
import math
K1, B, AVGDL, IDF = 1.2, 0.75, 60.0, 2.0
def w(tf, dl):
if tf == 0:
return 0.0
return IDF * tf * (K1 + 1) / (tf + K1 * (1 - B + B * dl / AVGDL))
print("k1 = %.1f, b = %.2f, avgdl = %.0f, idf = %.1f" % (K1, B, AVGDL,
IDF))
print("%6s %4s %8s %8s %8s %9s" % ("extra", "dl", "w(t1)", "w(t2)",
"total", "marginal"))
prev = None
for r in (0, 1, 2, 3, 5, 8, 12, 20, 40):
dl = 60 + r
total = math.fsum([w(1 + r, dl), w(1, dl)])
mg = "-" if prev is None else "%+.4f" % (total - prev[1])
if prev is not None and r - prev[0] > 1:
mg = "%+.4f" % ((total - prev[1]) / (r - prev[0]))
print("%6d %4d %8.4f %8.4f %8.4f %9s" % (r, dl, w(1 + r, dl),
w(1, dl), total, mg))
prev = (r, total)
cap = IDF * (K1 + 1)
print("ceiling of w(t1) as tf grows: idf (k1 + 1) = %.4f" % cap)
base = w(1, 60) * 2
new = w(1, 61) * 3
rep = math.fsum([w(2, 61), w(1, 61)])
print("one new matching term t3 : total %.4f (+%.4f)" % (new,
new - base))
print("one more repetition of t1: total %.4f (+%.4f)" % (rep,
rep - base))Single source and citation
Fan, Y. (2026). The Geometry of Citation Weight in Generative Search: Retrieval and Reranking Features and the Paragraph Topology of Highly Cited Bilingual Answer Pages (Version 1.0.0), Chapter 2: The Geometry of Hybrid Scoring: BM25 Saturation, Cosine Direction and Rank Fusion. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-260 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026260,
author = {Fan, Yafu},
title = {The Geometry of Citation Weight in Generative Search: Retrieval and Reranking Features and the Paragraph Topology of Highly Cited Bilingual Answer Pages},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Measuring Visibility: Why Keyword Stuffing Fails and Why Sources, Figures and Quotations Help
- Chapter 3 · Paragraph Length and the Point of the Cut: Chunking, Truncation and an Interior Optimum
- Chapter 4 · The Discount by Citation Position: Cumulated Gain, Position-Adjusted Word Count and the Sag in the Middle
- Chapter 5 · Verifiability: Why Sources Raise the Chance of Citation, and the Trap of Inferring Features from Outside
- Chapter 6 · The Paragraph Topology of a Bilingual Direct-Answer Page: Self-Contained Paragraphs, One Set of Numbers on Four Surfaces, and Compliant Structured Data
- Chapter 7 · Classical Sources East and West: the Three Gauges of the Mozi and Garfield's Citation Index
- Chapter 8 · Closing the Loop: an A/B Test of Answer-Paragraph Templates, the Citability Scorer and a Pre-Registered Test