What does Falsifiability and the Replication Crisis conclude, in numbers, about “The Remedies of Meta-Science: Pre-Registration, Registered Reports and Equivalence Tests”?
Answer: Pre-registration writes down what a theory forbids before the data are seen, bringing the false-positive rate back to its nominal level; it does not forbid multiple outcomes or interim analyses, only requires that they be stated in advance and the criteria adjusted. Registered Reports make publication independent of the result; the positive rate for first hypotheses is about 44 % in Registered Reports and about 96 % in the standard literature. Equivalence tests let "no difference" be shown. This series' decision rule has five verdicts; it controls the Type I error, but "90 % power" refers to reaching some verdict of benefit, and the strongest verdict appears only 2.5 % of the time when the true effect exactly equals the SESOI. The next chapter returns to classical Chinese scholarship to see what "seek truth from facts" and "without evidence there is no trust" meant in their original texts.
Scope of the numbers: This book is not medical, psychological or investment advice. Marked boxes separate published evidence, runnable Python calculations (14 programs, all checked), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 90
| true effect / SESOI | | V1 >=SESOI | V2 benefit | V3 harm | V4 no diff | V5 unclear | TOST 90% CI | ||
| -0.50 | | | 0.000 | 0.000 | 0.367 | 0.365 | 0.267 | | 0.490 |
| 0.00 | | | 0.000 | 0.025 | 0.025 | 0.800 | 0.150 | | 0.890 |
| 0.25 | | | 0.000 | 0.125 | 0.003 | 0.663 | 0.209 | | 0.776 |
| 0.50 | | | 0.000 | 0.367 | 0.000 | 0.365 | 0.267 | | 0.490 |
| 0.75 | | | 0.003 | 0.678 | 0.000 | 0.125 | 0.194 | | 0.202 |
| 1.00 | | | 0.025 | 0.875 | 0.000 | 0.025 | 0.075 | | 0.050 |
| 1.50 | | | 0.367 | 0.631 | 0.000 | 0.000 | 0.002 | | 0.001 |
| 2.00 | | | 0.900 | 0.100 | 0.000 | 0.000 | 0.000 | | 0.000 |
| check by simulation, true effect 0 SESOI: V1 0.000 | V2 0.025 | V3 0.026 | V4 0.799 | V5 0.150 | |||
| check by simulation, true effect 1 SESOI: V1 0.026 | V2 0.875 | V3 0.000 | V4 0.025 | V5 0.074 |
Show the full program
# The five-verdict rule of the 5D Library's Chapter 8 trials, and TOST (Lakens 2017), for
# a trial sized for 90 % power at the SESOI (= delta), two-sided alpha 0.05, so SE = delta
# / 3.24. Verdicts from the 95 % CI (lo, hi) of the effect, positive = benefit, checked in
# this order:
# V1 lo >= delta: benefit of at least the SESOI
# V2 lo > 0: benefit, size not shown to reach it
# V3 hi < 0: HARM
# V4 -delta <= lo and hi <= delta: no difference worth acting on
# V5 otherwise: unclear
import math, random
from statistics import NormalDist
N = NormalDist()
Z95, Z90 = N.inv_cdf(0.975), N.inv_cdf(0.90)
def verdict(est, se, delta, z=Z95):
lo, hi = est - z * se, est + z * se
if lo >= delta: return 0
if lo > 0: return 1
if hi < 0: return 2
if lo >= -delta and hi <= delta: return 3
return 4
def probs(theta, delta=1.0):
se = delta / (Z95 + Z90)
cuts = [(-math.inf, -Z95 * se, 2), (-Z95 * se, -delta + Z95 * se, 4),
(-delta + Z95 * se, delta - Z95 * se, 3), (delta - Z95 * se, Z95 * se, 4),
(Z95 * se, delta + Z95 * se, 1), (delta + Z95 * se, math.inf, 0)]
out = [0.0] * 5
for a, b, v in cuts:
out[v] += N.cdf((b - theta) / se) - N.cdf((a - theta) / se)
tost = (N.cdf((delta - 1.645 * se - theta) / se)
- N.cdf((-delta + 1.645 * se - theta) / se))
return out, tost
if __name__ == "__main__":
print("true effect / SESOI | V1 >=SESOI V2 benefit V3 harm V4 no diff V5 unclear"
" | TOST 90% CI")
for r in (-0.5, 0.0, 0.25, 0.5, 0.75, 1.0, 1.5, 2.0):
p, tost = probs(r)
cols = " ".join(f"{x:9.3f}" for x in p)
print(f" {r:5.2f} | " + cols + f" | {tost:9.3f}")
# check probs() against verdict() directly
rng, se = random.Random(355), 1 / (Z95 + Z90)
for r in (0.0, 1.0):
c = [0] * 5
for _ in range(100000):
c[verdict(rng.gauss(r, se), se, 1.0)] += 1
vs = " ".join(f"V{k + 1} {x / 1e5:.3f}" for k, x in enumerate(c))
print(f"check by simulation, true effect {r:.0f} SESOI: " + vs)Single source and citation
Fan, Y. (2026). Scientific Methodology, Falsifiability and the Mathematical Statistics of the Replication Crisis (Version 1.0.0), Chapter 6: The Remedies of Meta-Science: Pre-Registration, Registered Reports and Equivalence Tests. 5D Library. https://doi.org/10.5281/zenodo.23782104
DOI: 10.5281/zenodo.23782104 · Read the monograph · 5DL-B-033 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026033,
author = {Fan, Yafu},
title = {Scientific Methodology, Falsifiability and the Mathematical Statistics of the Replication Crisis},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.23782104},
url = {https://doi.org/10.5281/zenodo.23782104}
}Other chapter answers in this book
- Chapter 1 · Falsifiability and Research Programmes: What Counts as a Scientific Statement?
- Chapter 2 · Why Most Published Findings May Be False: Ioannidis's Derivation
- Chapter 3 · p-Hacking, Researcher Degrees of Freedom and the Garden of Forking Paths
- Chapter 4 · Publication Bias, the File Drawer and the Winner's Curse
- Chapter 5 · Large-Scale Replication: Results and Controversy
- Chapter 8 · Practice and a Pre-Registered Trial