What does Link Authority, Retrieval Truncation and Direct Answers conclude, in numbers, about “Practice and Falsification: the Copy Checker, a Publisher's Checklist and a Pre-Registered Test”?
Answer: Chapter 8 gives a publisher's checklist, a copy checker and a pre-registered test; detecting a moderate lift needs 1,081 runs per group. Marked boxes separate published evidence, runnable Python calculations (20 programs), classical sources and the author's hypotheses.
Scope of the numbers: Its parameters are illustrative; all calculations are generated by its programs; the book is not personalised legal, financial or business advice and makes no promise about citation rankings in any system. Chapter 1 computes PageRank and HITS with power iteration and a random surfer: in an illustrative knowledge base with damping factor 0.85, the mother page receives 0.3240, or 2.59 times the uniform share.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 1081
| z for alpha 1.9600, z for power 0.8416, design effect 1.4 | ||||
| p1 | lift | runs/arm | with clustering | queries/arm |
| 0.22 | 0.02 | 6,510 | 9,114 | 1,823 |
| 0.24 | 0.04 | 1,683 | 2,357 | 472 |
| 0.26 | 0.06 | 772 | 1,081 | 217 |
| 0.30 | 0.10 | 294 | 412 | 83 |
| with 1,500 runs per arm the minimum detectable lift is 0.048 |
Show the full program
# Chapter 8. p82: sample size for a pre-registered test of the book's
# core claim: rewriting a page into direct-answer form raises the share
# of generated answers that cite it. Unit: one (query, engine run).
# Control citation share p0 = 0.20 (illustrative); alpha = 0.05 two-
# sided, power 0.80, two-proportion z-test:
# n = (z_a sqrt(2 pbar qbar) + z_b sqrt(p0 q0 + p1 q1))^2 / (p1-p0)^2
# Runs cluster within queries: m = 5 runs per query, intra-query
# correlation 0.10, design effect 1 + (m - 1) * 0.10 = 1.4.
import math
from statistics import NormalDist
Z = NormalDist()
ZA = Z.inv_cdf(1 - 0.05 / 2)
ZB = Z.inv_cdf(0.80)
P0 = 0.20
DEFF = 1 + (5 - 1) * 0.10
def n_per_arm(p0, p1):
pb = (p0 + p1) / 2
num = (ZA * math.sqrt(2 * pb * (1 - pb))
+ ZB * math.sqrt(p0 * (1 - p0) + p1 * (1 - p1))) ** 2
return math.ceil(num / (p1 - p0) ** 2)
if __name__ == "__main__":
print("z for alpha {:.4f}, z for power {:.4f}, design effect {:.1f}"
.format(ZA, ZB, DEFF))
print(" p1 lift runs/arm with clustering queries/arm")
for p1 in (0.22, 0.24, 0.26, 0.30):
n = n_per_arm(P0, p1)
nc = math.ceil(n * DEFF)
print("{:5.2f} {:5.2f} {:9,d} {:15,d} {:11,d}".format(
p1, p1 - P0, n, nc, math.ceil(nc / 5)))
n = 1500
se = math.sqrt(2 * P0 * (1 - P0) / (n / DEFF))
print("with {:,} runs per arm the minimum detectable lift is {:.3f}"
.format(n, (ZA + ZB) * se))Single source and citation
Fan, Y. (2026). Hyperlink Authority Propagation in Machine-Readable Knowledge Bases, Retrieval-Truncation Games in Generative Answering and High-SNR Snippet Extraction (Version 1.0.0), Chapter 8: Practice and Falsification: the Copy Checker, a Publisher's Checklist and a Pre-Registered Test. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-176 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026176,
author = {Fan, Yafu},
title = {Hyperlink Authority Propagation in Machine-Readable Knowledge Bases, Retrieval-Truncation Games in Generative Answering and High-SNR Snippet Extraction},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Power Iteration and the Damping Factor: How Hyperlink Authority Flows Through a Knowledge Base
- Chapter 2 · The Mother Page: How Internal-Link Topology Concentrates Authority
- Chapter 3 · BM25 and Length Normalisation: the Term-Frequency Ledger of Stuffing, Padding and Direct Answers
- Chapter 4 · Top-k Truncation: the Probability that a Passage Enters the Context
- Chapter 5 · Competing for k Positions: Position Auctions, Homogenisation and the Commons
- Chapter 6 · Signal-to-Noise and Softmax Reranking: the Selection Gain of Direct-Answer Snippets
- Chapter 7 · The Classics Examined: Zheng Qiao's Classification and Bush's Associative Trails