关于「怎样量出相关:两两相关、Q 统计量、等效票数与错答撞车」,《相关误差下的多模型投票》给出的定量结论是什么?
结论:相关可以量,而且应该量。phi 系数与矩估计直接对应模型里的 \(\rho\),Q 统计量更敏感但刻度不同,同时判错的比例最直观。量出 \(\rho\) 之后,等效票数的上界 \(1/\rho\) 告诉店主还有多少独立信息可买;多选题还要再量一笔撞车率。下一章转向行动:既然相关来自共同原因,有哪些办法能把它拆掉,各能拆掉多少。
数据口径:书中参数为示意;全部计算由书中程序生成;本书是非个人化的经营分析,不构成经营建议。 书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库
| 600 items x 5 models, true mean accuracy 0.75 | |
| true rho p-hat phi Q both-wrong (1-p)^2 | |
| 0.0 | 0.7490 -0.0105 -0.0305 0.0610 0.0630 |
| 0.25 | 0.7630 0.2649 0.5747 0.1040 0.0562 |
| moment estimate of rho and the majority it predicts | |
| true rho rho-hat predicted maj observed maj | |
| 0.0 | -0.0102 0.8954 0.8950 |
| 0.25 | 0.2654 0.8196 0.8183 |
查看完整验算程序
# Chapter 3. p31: measuring error correlation from a labelled sample.
# Model: a synthetic item-by-model table of right (1) / wrong (0).
# Each of 600 items draws a shared difficulty theta ~ Beta(a, b) with
# mean 0.75 and correlation rho; five models answer it independently
# given theta (fixed seed). A second table has rho = 0. Estimators:
# pairwise phi (Pearson on 0/1), the Q statistic of Yule used by
# Kuncheva and Whitaker, the both-wrong rate against (1 - p)^2, and a
# moment estimate of rho from the spread of right answers per item,
# rho = (s^2 / (n p (1 - p)) - 1) / (n - 1). The estimate is then fed
# to the Beta-binomial majority of p21. All parameters illustrative.
import itertools
import math
import random
def lbeta(a, b):
return math.lgamma(a) + math.lgamma(b) - math.lgamma(a + b)
def ab(p, rho):
s = 1 / rho - 1
return p * s, (1 - p) * s
def bb_major(n, p, rho):
if rho == 0:
lp, lq = math.log(p), math.log(1 - p)
terms = [math.exp(math.lgamma(n + 1) - math.lgamma(k + 1)
- math.lgamma(n - k + 1) + k * lp
+ (n - k) * lq)
for k in range(n // 2 + 1, n + 1)]
return math.fsum(terms)
a, b = ab(p, rho)
terms = [math.exp(math.lgamma(n + 1) - math.lgamma(k + 1)
- math.lgamma(n - k + 1) + lbeta(k + a, n - k + b)
- lbeta(a, b))
for k in range(n // 2 + 1, n + 1)]
return math.fsum(terms)
def betacf(a, b, x):
tiny = 1e-300
c, d = 1.0, 1.0 - (a + b) * x / (a + 1)
d = 1.0 / (d if abs(d) > tiny else tiny)
h = d
for m in range(1, 400):
m2 = 2 * m
for num in (m * (b - m) * x / ((a + m2 - 1) * (a + m2)),
-(a + m) * (a + b + m) * x / ((a + m2) * (a + m2 + 1))):
d = 1.0 + num * d
d = 1.0 / (d if abs(d) > tiny else tiny)
c = 1.0 + num / c
c = c if abs(c) > tiny else tiny
h *= d * c
if abs(d * c - 1.0) < 1e-15:
break
return h
def betai(a, b, x):
if x <= 0:
return 0.0
if x >= 1:
return 1.0
lead = math.exp(a * math.log(x) + b * math.log(1 - x) - lbeta(a, b))
if x < (a + 1) / (a + b + 2):
return lead * betacf(a, b, x) / a
return 1.0 - lead * betacf(b, a, 1 - x) / b
def limit(p, rho):
if rho == 0:
return 1.0 if p > 0.5 else 0.0
a, b = ab(p, rho)
return 1.0 - betai(a, b, 0.5)
def table(rng, items, n, p, rho):
rows = []
for _ in range(items):
if rho > 0:
a, b = ab(p, rho)
th = rng.betavariate(a, b)
else:
th = p
rows.append([1 if rng.random() < th else 0 for _ in range(n)])
return rows
def stats(rows):
n = len(rows[0])
m = len(rows)
ph = math.fsum(math.fsum(r) for r in rows) / (m * n)
phis, qs, bw = [], [], []
for i, j in itertools.combinations(range(n), 2):
c = [[0, 0], [0, 0]]
for r in rows:
c[r[i]][r[j]] += 1
n11, n00, n10, n01 = c[1][1], c[0][0], c[1][0], c[0][1]
den = math.sqrt((n11 + n10) * (n01 + n00)
* (n11 + n01) * (n10 + n00))
phis.append((n11 * n00 - n01 * n10) / den)
qs.append((n11 * n00 - n01 * n10) / (n11 * n00 + n01 * n10))
bw.append(n00 / m)
ks = [math.fsum(r) for r in rows]
kbar = math.fsum(ks) / m
s2 = math.fsum((k - kbar) ** 2 for k in ks) / (m - 1)
rho = (s2 / (n * ph * (1 - ph)) - 1) / (n - 1)
maj = math.fsum(1.0 for k in ks if k > n / 2) / m
return ph, math.fsum(phis) / len(phis), math.fsum(qs) / len(qs), \
math.fsum(bw) / len(bw), rho, maj
if __name__ == "__main__":
rng = random.Random(20261113)
print("600 items x 5 models, true mean accuracy 0.75")
print(" true rho p-hat phi Q both-wrong (1-p)^2")
res = {}
for rho in (0.0, 0.25):
ph, phi, q, bw, rh, maj = stats(table(rng, 600, 5, 0.75, rho))
res[rho] = (ph, rh, maj)
print(" {:<8} {:.4f} {:.4f} {:.4f} {:.4f} {:.4f}".format(
rho, ph, phi, q, bw, (1 - ph) ** 2))
print("moment estimate of rho and the majority it predicts")
print(" true rho rho-hat predicted maj observed maj")
for rho in (0.0, 0.25):
ph, rh, maj = res[rho]
pred = bb_major(5, ph, max(rh, 0.0))
print(" {:<8} {:7.4f} {:13.4f} {:12.4f}".format(
rho, rh, pred, maj))唯一数据源与引用
范雅福(Fan, Y.). (2026). 《孔多塞陪审团定理与相关误差、多模型投票何时越投越准何时同源共错母典》第 3 章「怎样量出相关:两两相关、Q 统计量、等效票数与错答撞车」(版本 1.0.0). 5D Library. DOI 待发布
DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-309 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026309,
author = {Fan, Yafu},
title = {Condorcet's Jury Theorem with Correlated Errors: When Multi-Model Voting Gets More Accurate and When Shared Origins Make Everyone Wrong Together},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}