What does Regression to the Mean and the Cover Jinx conclude, in numbers, about “Shrinkage Estimation: James and Stein, Efron and Morris's Eighteen Batters, and Shrinkage Ranking”?
Answer: Chapter 4 explains shrinkage with Efron and Morris's eighteen batters: in a simplified version the total squared error of the raw averages is 3.53 times that of the shrunk estimates.
Scope of the numbers: Marked boxes separate published evidence, runnable Python calculations (22 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 3.53
| k 18 | grand mean 0.2654 | shrink factor c 0.2117 | ||
| player raw JS rest | player raw JS rest | ||||
| Clemente | 0.400 0.294 0.346 | | | F.Robinson | 0.378 0.289 0.298 |
| Howard | 0.356 0.285 0.276 | | | Johnstone | 0.333 0.280 0.222 |
| Berry | 0.311 0.275 0.273 | | | Spencer | 0.311 0.275 0.270 |
| Kessinger | 0.289 0.270 0.263 | | | Alvarado | 0.267 0.266 0.210 |
| Santo | 0.244 0.261 0.269 | | | Swoboda | 0.244 0.261 0.230 |
| Unser | 0.222 0.256 0.264 | | | Williams | 0.222 0.256 0.256 |
| Scott | 0.222 0.256 0.303 | | | Petrocelli | 0.222 0.256 0.264 |
| Rodriguez | 0.222 0.256 0.226 | | | Campaneris | 0.200 0.252 0.285 |
| Munson | 0.178 0.247 0.316 | | | Alvis | 0.156 0.242 0.200 |
| total squared error: raw 0.0753 | JS 0.0213 | ratio 3.53 | ||
| players whose JS estimate is closer: 16 of 18 | ||||
Show the full program
# Chapter 4. p41: Efron and Morris's 18 batters, shrunk.
# Data: hits in the first 45 at-bats of the 1970 season and the batting
# average over the rest of that season, for the 18 players of Efron and
# Morris (JASA, 1975), as distributed in the R package pscl (EfronMorris).
# A simplified James-Stein estimator (no arcsine transform):
# z_i = ybar + c (y_i - ybar), c = 1 - (k - 3) s2 / S,
# s2 = ybar (1 - ybar) / 45, S = sum of (y_i - ybar)^2.
# Error is measured against the rest-of-season average.
# 真实数据:埃夫隆与莫里斯的十八名击球手,前 45 次打数与赛季余下打击率。
import math
AB = 45
DATA = [("Clemente", 18, 0.346), ("F.Robinson", 17, 0.298),
("Howard", 16, 0.276), ("Johnstone", 15, 0.222),
("Berry", 14, 0.273), ("Spencer", 14, 0.270),
("Kessinger", 13, 0.263), ("Alvarado", 12, 0.210),
("Santo", 11, 0.269), ("Swoboda", 11, 0.230),
("Unser", 10, 0.264), ("Williams", 10, 0.256),
("Scott", 10, 0.303), ("Petrocelli", 10, 0.264),
("Rodriguez", 10, 0.226), ("Campaneris", 9, 0.285),
("Munson", 8, 0.316), ("Alvis", 7, 0.200)]
y = [h / AB for _, h, _ in DATA]
p = [r for _, _, r in DATA]
k = len(y)
ybar = math.fsum(y) / k
s2 = ybar * (1 - ybar) / AB
S = math.fsum((v - ybar) ** 2 for v in y)
c = 1 - (k - 3) * s2 / S
z = [ybar + c * (v - ybar) for v in y]
if __name__ == "__main__":
print("k {} grand mean {:.4f} shrink factor c {:.4f}"
.format(k, ybar, c))
print("player raw JS rest | player raw JS rest")
for i in range(0, k, 2):
cells = []
for j in (i, i + 1):
cells.append("{:<11} {:.3f} {:.3f} {:.3f}".format(
DATA[j][0], y[j], z[j], p[j]))
print(" | ".join(cells))
e_raw = math.fsum((a - b) ** 2 for a, b in zip(y, p))
e_js = math.fsum((a - b) ** 2 for a, b in zip(z, p))
better = len([1 for a, b, t in zip(y, z, p) if abs(b - t) < abs(a - t)])
print("total squared error: raw {:.4f} JS {:.4f} ratio {:.2f}"
.format(e_raw, e_js, e_raw / e_js))
print("players whose JS estimate is closer: {} of {}".format(better, k))Single source and citation
Fan, Y. (2026). Regression to the Mean and the Cover Jinx: Posterior Calibration of Fortune Reports and Serial Independence in the Tale of the Lost Horse (Version 1.0.0), Chapter 4: Shrinkage Estimation: James and Stein, Efron and Morris's Eighteen Batters, and Shrinkage Ranking. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-202 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026202,
author = {Fan, Yafu},
title = {Regression to the Mean and the Cover Jinx: Posterior Calibration of Fortune Reports and Serial Independence in the Tale of the Lost Horse},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · A Conditional Expectation with Slope Below One: Why Top Stores Slip the Next Year
- Chapter 2 · The Flight Instructor's Illusion: Why Criticism Looks More Effective than Praise
- Chapter 3 · Splitting Variance into Skill and Luck: Repeated Measures, Intraclass Correlation and Reliability
- Chapter 5 · The Cover Jinx: Select at the Peak and You Will See a Fall
- Chapter 6 · Posterior Calibration of Fortune Reports: Buying at the Low Point, Barnum Statements and the Brier Score
- Chapter 7 · Sources East and West: the Huainanzi's Old Man Who Lost His Horse and Galton's 1886 "Regression towards Mediocrity"
- Chapter 8 · Closing the Loop: Annual Reviews by Shrinkage Ranking, Copy without Causal Overstatement, and a Pre-registered Plan