What does Falsifiability and the Replication Crisis conclude, in numbers, about “Practice and a Pre-Registered Trial”?
Answer: The four-step checklist asks in turn whether a claim was falsifiable in advance, whether it went through multiple comparisons and optional stopping, whether the sample was so small that the effect must be exaggerated, and whether it has been independently replicated with interests disclosed. Applied to three fictional claims, it clearly separates the case report, the supplement advertisement and the pre-registered trial. Applied to the series itself, it shows that the series' protocols meet the requirements of pre-registration only on paper (they have not been registered), and that there are no data, the effect-size assumptions lack a prior basis, the text is AI-drafted, the author has a commercial interest, and there has been no independent replication. The pre-registered trial has \(d'\) as its single primary outcome, 176 analysable and 208 recruited per arm, a decision rule written as code, and will be published whatever the result.
Scope of the numbers: This book is not medical, psychological or investment advice. Marked boxes separate published evidence, runnable Python calculations (14 programs, all checked), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 176, 208
| control arm: observed d' mean 0.96 (true 1.00), SD between people 0.72 | |
| true d' raised by 0.25 -> observed difference 0.23 (hits 0.67 | 0.71, false alarms 0.33 -> 0.29) |
| only more sceptical (c lowered by 0.4) -> d' difference -0.02 (hits 0.78, false alarms 0.46) | |
| people per arm, 90 % power, alpha 0.05 | observed-d' SESOI 0.20 | 0.25 | 0.30 | |
| SD of observed d' | 0.60 | 191 | 123 | 86 |
| SD of observed d' | 0.72 | 274 | 176 | 123 |
| SD of observed d' | 0.90 | 427 | 274 | 191 |
| chosen: SESOI 0.25 (observed d'), SD 0.72 | 176 analysable per arm |
| allowing 15 % exclusions (attention checks, incomplete sessions) -> recruit 208 per arm, 416 in all |
Show the full program
# Sample size for the Chapter 8 trial. Primary outcome: discrimination d' when judging 24
# news reports (12 contain an exaggerated claim from a methodological flaw, 12 are
# accurate), with the log-linear correction (add 0.5 to each count, 1 to each total;
# Hautus 1995). ASSUMED, not taken from any study: true d' in the control arm ~ N(1.0,
# 0.5) between people, criterion c ~ N(0, 0.5).
import math, random
from statistics import NormalDist, mean, stdev
N = NormalDist()
def observed_dprime(rng, d, c, n=12):
h = sum(rng.random() < N.cdf(d / 2 - c) for _ in range(n))
f = sum(rng.random() < N.cdf(-d / 2 - c) for _ in range(n))
return N.inv_cdf((h + 0.5) / (n + 1)) - N.inv_cdf((f + 0.5) / (n + 1)), h / n, f / n
def arm(rng, shift_d=0.0, shift_c=0.0, people=20000):
out = [observed_dprime(rng, rng.gauss(1.0 + shift_d, 0.5), rng.gauss(shift_c, 0.5))
for _ in range(people)]
return [o[0] for o in out], mean(o[1] for o in out), mean(o[2] for o in out)
def per_arm(sd, delta, alpha=0.05, power=0.90):
z = N.inv_cdf(1 - alpha / 2) + N.inv_cdf(power)
return math.ceil(2 * z ** 2 * sd ** 2 / delta ** 2) + 1
if __name__ == "__main__":
rng = random.Random(1995)
b, hb, fb = arm(rng)
a, ha, fa = arm(rng, shift_d=0.25)
s, hs, fs = arm(rng, shift_c=-0.4)
sd = stdev(b)
print(f"control arm: observed d' mean {mean(b):.2f} (true 1.00),"
f" SD between people {sd:.2f}")
print(f"true d' raised by 0.25 -> observed difference {mean(a) - mean(b):.2f}"
f" (hits {hb:.2f} -> {ha:.2f}, false alarms {fb:.2f} -> {fa:.2f})")
print(f"only more sceptical (c lowered by 0.4) ->"
f" d' difference {mean(s) - mean(b):+.2f}"
f" (hits {hs:.2f}, false alarms {fs:.2f})")
print("people per arm, 90 % power, alpha 0.05 | observed-d' SESOI 0.20 | 0.25 | 0.30")
for v in (0.60, round(sd, 2), 0.90):
ns = " |".join(f"{per_arm(v, dl):5d}" for dl in (0.20, 0.25, 0.30))
print(f" SD of observed d' = {v:.2f} {'':16s}|" + ns)
n = per_arm(round(sd, 2), 0.25)
rec = math.ceil(n / 0.85)
print(f"chosen: SESOI 0.25 (observed d'), SD {sd:.2f} -> {n} analysable per arm")
print(f"allowing 15 % exclusions (attention checks, incomplete sessions)"
f" -> recruit {rec} per arm,"
f" {2 * rec} in all")Single source and citation
Fan, Y. (2026). Scientific Methodology, Falsifiability and the Mathematical Statistics of the Replication Crisis (Version 1.0.0), Chapter 8: Practice and a Pre-Registered Trial. 5D Library. https://doi.org/10.5281/zenodo.23782104
DOI: 10.5281/zenodo.23782104 · Read the monograph · 5DL-B-033 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026033,
author = {Fan, Yafu},
title = {Scientific Methodology, Falsifiability and the Mathematical Statistics of the Replication Crisis},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.23782104},
url = {https://doi.org/10.5281/zenodo.23782104}
}Other chapter answers in this book
- Chapter 1 · Falsifiability and Research Programmes: What Counts as a Scientific Statement?
- Chapter 2 · Why Most Published Findings May Be False: Ioannidis's Derivation
- Chapter 3 · p-Hacking, Researcher Degrees of Freedom and the Garden of Forking Paths
- Chapter 4 · Publication Bias, the File Drawer and the Winner's Curse
- Chapter 5 · Large-Scale Replication: Results and Controversy
- Chapter 6 · The Remedies of Meta-Science: Pre-Registration, Registered Reports and Equivalence Tests