What does Falsifiability and the Replication Crisis conclude, in numbers, about “Why Most Published Findings May Be False: Ioannidis's Derivation”?
Answer: Ioannidis applied the Bayesian reasoning of diagnostic tests to scientific findings: the positive predictive value is fixed by the pre-study odds \(R\), the power and the significance level, and bias \(u\) lowers it further. The framework's way of writing the formula is equivalent to the original; with power 0.20, \(R = 0.1\) and \(u = 0.20\) the PPV is about 13.0 %. Goodman and Greenland pointed out that dichotomising \(p\)-values hides differences in the strength of evidence and that "most are false" depends on assumptions; Jager and Leek's empirical estimate (about 14 %) rests on contested assumptions of its own. The next chapter looks at where bias \(u\) comes from: the researcher's freedom in analysing data.
Scope of the numbers: This book is not medical, psychological or investment advice. Marked boxes separate published evidence, runnable Python calculations (14 programs, all checked), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. printed values that match the answer: 0.1, 0.20
| largest difference between the two forms over 11970 parameter sets: 1.1e-15 | |||
| power 0.20, R = 0.1: PPV = 0.1304 (u = 0.2), | 0.1161 (u = 0.3), | 0.2857 (u = 0.0) | |
| Table 4 of Ioannidis 2005, recomputed: power R u PPV (paper) | |||
| 0.80 | 1.0000 | 0.10 | 0.8497 (0.85) |
| 0.95 | 2.0000 | 0.30 | 0.8521 (0.85) |
| 0.80 | 0.3333 | 0.40 | 0.4055 (0.41) |
| 0.20 | 0.2000 | 0.20 | 0.2308 (0.23) |
| 0.20 | 0.2000 | 0.80 | 0.1718 (0.17) |
| 0.80 | 0.1000 | 0.30 | 0.2043 (0.20) |
| 0.20 | 0.1000 | 0.30 | 0.1161 (0.12) |
| 0.20 | 0.0010 | 0.80 | 0.0010 (0.0010) |
| 0.20 | 0.0010 | 0.20 | 0.0015 (0.0015) |
| n independent teams, no bias, R = 0.1: | n = 1 | 5 | 10 |
| power 0.20 | 0.286 | 0.229 | 0.182 |
| power 0.80 | 0.615 | 0.306 | 0.199 |
Key formula
Show the full program
# Positive predictive value of a "research finding" (Ioannidis 2005, PLoS Med 2: e124). R
# = pre-study odds of a true relationship among those probed, 1 - beta = power, alpha =
# 0.05, u = share of analyses that would not have been findings but are reported as
# findings through bias.
def ppv_ioannidis(power, R, u, alpha=0.05):
beta = 1 - power
return (((1 - beta) * R + u * beta * R)
/ (R + alpha - beta * R + u - u * alpha + u * beta * R))
def ppv_author(power, R, u, alpha=0.05):
# the framework's form: [(1-b)R + u b R] / [(1-b)R + alpha + u b R + u(1-alpha)]
beta = 1 - power
num = (1 - beta) * R + u * beta * R
return num / ((1 - beta) * R + alpha + u * beta * R + u * (1 - alpha))
def ppv_teams(power, R, n, alpha=0.05):
# n independent teams probe the same question; a "finding" if at least one is
# significant
beta = 1 - power
return R * (1 - beta ** n) / (R + 1 - (1 - alpha) ** n - R * beta ** n)
if __name__ == "__main__":
grid = [(p / 20, r / 10, u / 20) for p in range(1, 20) for r in range(1, 31)
for u in range(21)]
diff = max(abs(ppv_ioannidis(*g) - ppv_author(*g)) for g in grid)
print(f"largest difference between the two forms over {len(grid)} parameter sets:"
f" {diff:.1e}")
vals = ", ".join(f"{ppv_ioannidis(0.20, 0.1, u):.4f} (u = {u})"
for u in (0.2, 0.3, 0.0))
print("power 0.20, R = 0.1: PPV = " + vals)
print("Table 4 of Ioannidis 2005, recomputed: power R u PPV (paper)")
rows = [(0.80, 1, 0.10, "0.85"), (0.95, 2, 0.30, "0.85"), (0.80, 1 / 3, 0.40, "0.41"),
(0.20, 1 / 5, 0.20, "0.23"), (0.20, 1 / 5, 0.80, "0.17"),
(0.80, 1 / 10, 0.30, "0.20"),
(0.20, 1 / 10, 0.30, "0.12"), (0.20, 1 / 1000, 0.80, "0.0010"),
(0.20, 1 / 1000, 0.20, "0.0015")]
for pw, R, u, paper in rows:
f = ppv_ioannidis(pw, R, u)
print(f"{'':39s}{pw:.2f} {R:7.4f} {u:.2f} {f:.4f} ({paper})")
print("n independent teams, no bias, R = 0.1: n = 1 5 10")
for pw in (0.20, 0.80):
vals = " ".join(f"{ppv_teams(pw, 0.1, n):.3f}" for n in (1, 5, 10))
print(f" power {pw:.2f}{'':27s}" + vals)Single source and citation
Fan, Y. (2026). Scientific Methodology, Falsifiability and the Mathematical Statistics of the Replication Crisis (Version 1.0.0), Chapter 2: Why Most Published Findings May Be False: Ioannidis's Derivation. 5D Library. https://doi.org/10.5281/zenodo.23782104
DOI: 10.5281/zenodo.23782104 · Read the monograph · 5DL-B-033 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026033,
author = {Fan, Yafu},
title = {Scientific Methodology, Falsifiability and the Mathematical Statistics of the Replication Crisis},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.23782104},
url = {https://doi.org/10.5281/zenodo.23782104}
}Other chapter answers in this book
- Chapter 1 · Falsifiability and Research Programmes: What Counts as a Scientific Statement?
- Chapter 3 · p-Hacking, Researcher Degrees of Freedom and the Garden of Forking Paths
- Chapter 4 · Publication Bias, the File Drawer and the Winner's Curse
- Chapter 5 · Large-Scale Replication: Results and Controversy
- Chapter 6 · The Remedies of Meta-Science: Pre-Registration, Registered Reports and Equivalence Tests
- Chapter 8 · Practice and a Pre-Registered Trial