Scientific Methodology, Falsifiability and the Mathematical Statistics of the Replication Crisis: chapter answers
A supplement is "clinically proven", a case report says conversions tripled, a paper reports p < 0.05: why should we believe a claim? This book, the methodological charter of 5D Library, answers with the philosophy of science and mathematical statistics. It covers Popper's falsifiability and "empirical content" (the more possible worlds a statement forbids, the more it says, as in Bar-Hillel and Carnap's semantic information; a theory that forbids nothing in advance carries no information and can be supported by no observation), a Bayesian reading of the Duhem–Quine thesis and Lakatos's progr…
DOI 10.5281/zenodo.23782104 · Read the monograph · 中文版
- Chapter 1 · Falsifiability and Research Programmes: What Counts as a Scientific Statement?
The first case resembles the historical example Dorling discussed: a well-tested theory (0.90 beforehand) with a less reliable auxiliary (0.60). After the failure the theory falls only to 0.783, while the auxiliary drop… - Chapter 2 · Why Most Published Findings May Be False: Ioannidis's Derivation
Ioannidis applied the Bayesian reasoning of diagnostic tests to scientific findings: the positive predictive value is fixed by the pre-study odds $R$, the power and the significance level, and bias $u$ lowers it further… - Chapter 3 · p-Hacking, Researcher Degrees of Freedom and the Garden of Forking Paths
In data with no true effect, choosing among dependent variables, adding observations, adding covariates and dropping conditions each raise the false-positive rate from 5 % to roughly 8 % to 13 %, and together to about s… - Chapter 4 · Publication Bias, the File Drawer and the Winner's Curse
Journals publish mainly significant results, and this filtering has barely changed since 1959. The fail-safe N answers only "does an effect exist?" and assumes the drawer averages zero effect; in this chapter's simulati… - Chapter 5 · Large-Scale Replication: Results and Controversy
Three large replication projects found same-direction significance rates of 36.1 % (psychology, 35/97), 61 % (economics, 11/18) and 62 % (social-science experiments in Nature and Science, 13/21), with replication effect… - Chapter 6 · The Remedies of Meta-Science: Pre-Registration, Registered Reports and Equivalence Tests
Pre-registration writes down what a theory forbids before the data are seen, bringing the false-positive rate back to its nominal level; it does not forbid multiple outcomes or interim analyses, only requires that they… - Chapter 8 · Practice and a Pre-Registered Trial
The four-step checklist asks in turn whether a claim was falsifiable in advance, whether it went through multiple comparisons and optional stopping, whether the sample was so small that the effect must be exaggerated, a…
Illustrative model calculation, not a live quote