What does Chunking and Blending conclude, in numbers, about “The Quarterly Tuning Loop and a Plan for Refutation”?
Answer: Chapter 8 gives a quarterly tuning routine for knowledge-base chunking, the distilled "Chunk-and-Blend Grid Optimizer", a pre-registered test with sample sizes from Connor's formula (the exact McNemar test reaches an actual power of 0.762) and three conditions that would overturn the core claim. Boxes separate published evidence, runnable Python calculations (17 programs), classical sources and the author's hypotheses.
Scope of the numbers: Implementing BM25 and a concept-level bag-of-words cosine in standard-library programs on fixed-seed synthetic corpora, the book corrects the blueprint in three places. Parameters are illustrative and all numbers come from the book's programs, which read and write no file and connect to no live knowledge base.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.762
| psi | delta=0.02 | delta=0.03 | delta=0.05 |
| 0.06 | 1175 | 521 | 186 |
| 0.10 | 1960 | 870 | 312 |
| 0.20 | 3923 | 1742 | 626 |
| check psi=0.10 delta=0.03 n=870: simulated power 0.762 |
Show the full program
# Chapter 8. p82: how many labelled queries a paired A/B of two configs
# needs. Model: each query is scored under both configs (paired). With
# discordant share psi and true recall gap delta, Connor (1987) gives
# n = (z_a sqrt(psi) + z_b sqrt(psi - delta^2))^2 / delta^2, two-sided
# alpha 0.05, power 0.8. Check: 1000 simulated trials (seed 20261113)
# of an exact McNemar test at one cell.
import math
import random
from statistics import NormalDist
ZA, ZB = NormalDist().inv_cdf(0.975), NormalDist().inv_cdf(0.8)
def connor(psi, delta):
num = ZA * math.sqrt(psi) + ZB * math.sqrt(psi - delta * delta)
return math.ceil(num * num / (delta * delta))
def mcnemar_p(b, c):
m, k = b + c, min(b, c)
tail = math.fsum(math.comb(m, i) for i in range(k + 1)) / 2 ** m
return min(1.0, 2 * tail)
print(" psi delta=0.02 delta=0.03 delta=0.05")
for psi in (0.06, 0.1, 0.2):
print("%5.2f %10d %10d %10d" % (
psi, connor(psi, 0.02), connor(psi, 0.03), connor(psi, 0.05)))
psi, delta = 0.1, 0.03
n = connor(psi, delta)
pb, pc = (psi + delta) / 2, (psi - delta) / 2
rng, rej = random.Random(20261113), 0
for _ in range(1000):
b = c = 0
for _ in range(n):
u = rng.random()
if u < pb:
b += 1
elif u < pb + pc:
c += 1
rej += mcnemar_p(b, c) < 0.05
print("check psi=%.2f delta=%.2f n=%d: simulated power %.3f" % (
psi, delta, n, rej / 1000))Single source and citation
Fan, Y. (2026). Grid Search over Knowledge-Base Chunk Boundaries: Overlap Ratios and the Hybrid Weighting of Vector and Term-Frequency Retrieval (Version 1.0.0), Chapter 8: The Quarterly Tuning Loop and a Plan for Refutation. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-261 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026261,
author = {Fan, Yafu},
title = {Grid Search over Knowledge-Base Chunk Boundaries: Overlap Ratios and the Hybrid Weighting of Vector and Term-Frequency Retrieval},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Inverted U: Chunk Length and Recall under a Reading Budget
- Chapter 2 · BM25 from Scratch: Inverse Document Frequency, Saturation and Length Normalisation
- Chapter 3 · Bag-of-Words Cosine and Dilution in Long Chunks: Which Retriever Recognises Which Query
- Chapter 4 · Overlap: Exact Odds of a Boundary Cut and the Knee of Diminishing Returns
- Chapter 5 · The Blend: How the Best Weight Moves with the Query Mix
- Chapter 6 · The Joint Grid and the Winner's Curse: Why the Held-Out Test Cannot Be Skipped
- Chapter 7 · Cook Ding and the Phaedrus: Two Ancient Arguments for Cutting at the Natural Joints