What does Calibration and Collective Intelligence conclude, in numbers, about “Signal Detection Theory: Seeing Clearly and Where to Set the Threshold”?

Answer: The first three judges have almost the same sensitivity (\(d'\) about 2); the difference lies entirely in the threshold. The neutral judge has \(c = 0\) and \(\beta = 1\); the liberal one has \(c = -0.70\) and a hit rate of 95 %, paid for with 40 % false alarms; the conservative one has \(c = +1.00\), almost never raises a false alarm (2.3 %) and misses half the signals. Judged by hit rate alone, the liberal judge looks "best"; judged by false alarms alone, the conservative one does; in fact all three see equally well. The fourth judge has a 90 % hit rate, which looks good, but a 69 % false-alarm rate and \(d'\) of only 0.79: he is mostly saying "yes" to everything.

Scope of the numbers: This book is not counselling, medical or investment advice. Marked boxes separate published evidence, runnable Python calculations (16 programs, all checked), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: -0.70, 0.79, 1.00

observer | H FA | d' c beta
neutral | 0.840 0.160 |1.99 +0.00 1.00
liberal (says yes a lot) | 0.950 0.400 |1.90 -0.70 0.27
conservative | 0.500 0.023 |2.00 +1.00 7.32
weak but liberal | 0.900 0.690 |0.79 -0.89 0.50
ROC d'0.5: AUC = 0.638; (FA, H) at criteria -1, 0, 1, 2:
(0.84, 0.93) (0.50, 0.69) (0.16, 0.31) (0.02, 0.07)
ROC d'1.0: AUC = 0.760; (FA, H) at criteria -1, 0, 1, 2:
(0.84, 0.98) (0.50, 0.84) (0.16, 0.50) (0.02, 0.16)
ROC d'2.0: AUC = 0.921; (FA, H) at criteria -1, 0, 1, 2:
(0.84, 1.00) (0.50, 0.98) (0.16, 0.84) (0.02, 0.50)

Key formula

\[d' = z(H) - z(FA), \qquad c = -\frac{z(H) + z(FA)}{2}, \qquad \beta = e^{c\,d'}.\]
Show the full program
# Equal-variance Gaussian signal detection: noise N(0, 1), signal N(d', 1).
# d' = z(H) - z(FA), c = -(z(H) + z(FA)) / 2, beta = exp(c d'), AUC = Phi(d' / sqrt 2).
import math
from statistics import NormalDist

N = NormalDist()

def sdt(h, fa):
    zh, zf = N.inv_cdf(h), N.inv_cdf(fa)
    d = zh - zf
    c = -(zh + zf) / 2 + 0.0
    return d, c, math.exp(c * d)

def roc(d, crits):
    return [(1 - N.cdf(k), 1 - N.cdf(k - d)) for k in crits]

if __name__ == "__main__":
    print("observer                 |   H     FA  |   d'      c    beta")
    for name, h, fa in (("neutral", 0.84, 0.16), ("liberal (says yes a lot)", 0.95, 0.40),
                        ("conservative", 0.50, 0.023), ("weak but liberal", 0.90, 0.69)):
        d, c, b = sdt(h, fa)
        print(f"{name:24s} | {h:.3f} {fa:.3f} | {d:5.2f} {c:+6.2f} {b:6.2f}")
    for d in (0.5, 1.0, 2.0):
        pts = roc(d, (-1.0, 0.0, 1.0, 2.0))
        auc = N.cdf(d / math.sqrt(2))
        print(f"ROC d' = {d}: AUC = {auc:.3f}; (FA, H) at criteria -1, 0, 1, 2:")
        print("    " + "  ".join(f"({f:.2f}, {h:.2f})" for f, h in pts))

Single source and citation

Fan, Y. (2026). Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence (Version 1.0.0), Chapter 4: Signal Detection Theory: Seeing Clearly and Where to Set the Threshold. 5D Library. https://doi.org/10.5281/zenodo.23764998

DOI: 10.5281/zenodo.23764998 · Read the monograph · 5DL-B-026 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026026,
  author    = {Fan, Yafu},
  title     = {Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.23764998},
  url       = {https://doi.org/10.5281/zenodo.23764998}
}

Other chapter answers in this book