What does Chunking and Blending conclude, in numbers, about “The Quarterly Tuning Loop and a Plan for Refutation”?

Answer: Chapter 8 gives a quarterly tuning routine for knowledge-base chunking, the distilled "Chunk-and-Blend Grid Optimizer", a pre-registered test with sample sizes from Connor's formula (the exact McNemar test reaches an actual power of 0.762) and three conditions that would overturn the core claim. Boxes separate published evidence, runnable Python calculations (17 programs), classical sources and the author's hypotheses.

Scope of the numbers: Implementing BM25 and a concept-level bag-of-words cosine in standard-library programs on fixed-seed synthetic corpora, the book corrects the blueprint in three places. Parameters are illustrative and all numbers come from the book's programs, which read and write no file and connect to no live knowledge base.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.762

psidelta=0.02delta=0.03delta=0.05
0.061175521186
0.101960870312
0.2039231742626
check psi=0.10 delta=0.03 n=870: simulated power 0.762
Show the full program
# Chapter 8. p82: how many labelled queries a paired A/B of two configs
# needs. Model: each query is scored under both configs (paired). With
# discordant share psi and true recall gap delta, Connor (1987) gives
# n = (z_a sqrt(psi) + z_b sqrt(psi - delta^2))^2 / delta^2, two-sided
# alpha 0.05, power 0.8. Check: 1000 simulated trials (seed 20261113)
# of an exact McNemar test at one cell.
import math
import random
from statistics import NormalDist

ZA, ZB = NormalDist().inv_cdf(0.975), NormalDist().inv_cdf(0.8)


def connor(psi, delta):
    num = ZA * math.sqrt(psi) + ZB * math.sqrt(psi - delta * delta)
    return math.ceil(num * num / (delta * delta))


def mcnemar_p(b, c):
    m, k = b + c, min(b, c)
    tail = math.fsum(math.comb(m, i) for i in range(k + 1)) / 2 ** m
    return min(1.0, 2 * tail)


print("  psi  delta=0.02  delta=0.03  delta=0.05")
for psi in (0.06, 0.1, 0.2):
    print("%5.2f  %10d  %10d  %10d" % (
        psi, connor(psi, 0.02), connor(psi, 0.03), connor(psi, 0.05)))
psi, delta = 0.1, 0.03
n = connor(psi, delta)
pb, pc = (psi + delta) / 2, (psi - delta) / 2
rng, rej = random.Random(20261113), 0
for _ in range(1000):
    b = c = 0
    for _ in range(n):
        u = rng.random()
        if u < pb:
            b += 1
        elif u < pb + pc:
            c += 1
    rej += mcnemar_p(b, c) < 0.05
print("check psi=%.2f delta=%.2f n=%d: simulated power %.3f" % (
    psi, delta, n, rej / 1000))

Single source and citation

Fan, Y. (2026). Grid Search over Knowledge-Base Chunk Boundaries: Overlap Ratios and the Hybrid Weighting of Vector and Term-Frequency Retrieval (Version 1.0.0), Chapter 8: The Quarterly Tuning Loop and a Plan for Refutation. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-261 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026261,
  author    = {Fan, Yafu},
  title     = {Grid Search over Knowledge-Base Chunk Boundaries: Overlap Ratios and the Hybrid Weighting of Vector and Term-Frequency Retrieval},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book