关于「怎样量出相关:两两相关、Q 统计量、等效票数与错答撞车」,《相关误差下的多模型投票》给出的定量结论是什么?

结论:相关可以量,而且应该量。phi 系数与矩估计直接对应模型里的 \(\rho\),Q 统计量更敏感但刻度不同,同时判错的比例最直观。量出 \(\rho\) 之后,等效票数的上界 \(1/\rho\) 告诉店主还有多少独立信息可买;多选题还要再量一笔撞车率。下一章转向行动:既然相关来自共同原因,有哪些办法能把它拆掉,各能拆掉多少。

数据口径:书中参数为示意;全部计算由书中程序生成;本书是非个人化的经营分析,不构成经营建议。 书中以标注框区分已发表的证据、可运行的 Python 计算(19 段程序)、原典与作者的假说。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库

600 items x 5 models, true mean accuracy 0.75
true rho p-hat phi Q both-wrong (1-p)^2
0.00.7490 -0.0105 -0.0305 0.0610 0.0630
0.250.7630 0.2649 0.5747 0.1040 0.0562
moment estimate of rho and the majority it predicts
true rho rho-hat predicted maj observed maj
0.0-0.0102 0.8954 0.8950
0.250.2654 0.8196 0.8183
查看完整验算程序
# Chapter 3. p31: measuring error correlation from a labelled sample.
# Model: a synthetic item-by-model table of right (1) / wrong (0).
# Each of 600 items draws a shared difficulty theta ~ Beta(a, b) with
# mean 0.75 and correlation rho; five models answer it independently
# given theta (fixed seed). A second table has rho = 0. Estimators:
# pairwise phi (Pearson on 0/1), the Q statistic of Yule used by
# Kuncheva and Whitaker, the both-wrong rate against (1 - p)^2, and a
# moment estimate of rho from the spread of right answers per item,
# rho = (s^2 / (n p (1 - p)) - 1) / (n - 1). The estimate is then fed
# to the Beta-binomial majority of p21. All parameters illustrative.
import itertools
import math
import random



def lbeta(a, b):
    return math.lgamma(a) + math.lgamma(b) - math.lgamma(a + b)


def ab(p, rho):
    s = 1 / rho - 1
    return p * s, (1 - p) * s


def bb_major(n, p, rho):
    if rho == 0:
        lp, lq = math.log(p), math.log(1 - p)
        terms = [math.exp(math.lgamma(n + 1) - math.lgamma(k + 1)
                          - math.lgamma(n - k + 1) + k * lp
                          + (n - k) * lq)
                 for k in range(n // 2 + 1, n + 1)]
        return math.fsum(terms)
    a, b = ab(p, rho)
    terms = [math.exp(math.lgamma(n + 1) - math.lgamma(k + 1)
                      - math.lgamma(n - k + 1) + lbeta(k + a, n - k + b)
                      - lbeta(a, b))
             for k in range(n // 2 + 1, n + 1)]
    return math.fsum(terms)


def betacf(a, b, x):
    tiny = 1e-300
    c, d = 1.0, 1.0 - (a + b) * x / (a + 1)
    d = 1.0 / (d if abs(d) > tiny else tiny)
    h = d
    for m in range(1, 400):
        m2 = 2 * m
        for num in (m * (b - m) * x / ((a + m2 - 1) * (a + m2)),
                    -(a + m) * (a + b + m) * x / ((a + m2) * (a + m2 + 1))):
            d = 1.0 + num * d
            d = 1.0 / (d if abs(d) > tiny else tiny)
            c = 1.0 + num / c
            c = c if abs(c) > tiny else tiny
            h *= d * c
        if abs(d * c - 1.0) < 1e-15:
            break
    return h


def betai(a, b, x):
    if x <= 0:
        return 0.0
    if x >= 1:
        return 1.0
    lead = math.exp(a * math.log(x) + b * math.log(1 - x) - lbeta(a, b))
    if x < (a + 1) / (a + b + 2):
        return lead * betacf(a, b, x) / a
    return 1.0 - lead * betacf(b, a, 1 - x) / b


def limit(p, rho):
    if rho == 0:
        return 1.0 if p > 0.5 else 0.0
    a, b = ab(p, rho)
    return 1.0 - betai(a, b, 0.5)


def table(rng, items, n, p, rho):
    rows = []
    for _ in range(items):
        if rho > 0:
            a, b = ab(p, rho)
            th = rng.betavariate(a, b)
        else:
            th = p
        rows.append([1 if rng.random() < th else 0 for _ in range(n)])
    return rows


def stats(rows):
    n = len(rows[0])
    m = len(rows)
    ph = math.fsum(math.fsum(r) for r in rows) / (m * n)
    phis, qs, bw = [], [], []
    for i, j in itertools.combinations(range(n), 2):
        c = [[0, 0], [0, 0]]
        for r in rows:
            c[r[i]][r[j]] += 1
        n11, n00, n10, n01 = c[1][1], c[0][0], c[1][0], c[0][1]
        den = math.sqrt((n11 + n10) * (n01 + n00)
                        * (n11 + n01) * (n10 + n00))
        phis.append((n11 * n00 - n01 * n10) / den)
        qs.append((n11 * n00 - n01 * n10) / (n11 * n00 + n01 * n10))
        bw.append(n00 / m)
    ks = [math.fsum(r) for r in rows]
    kbar = math.fsum(ks) / m
    s2 = math.fsum((k - kbar) ** 2 for k in ks) / (m - 1)
    rho = (s2 / (n * ph * (1 - ph)) - 1) / (n - 1)
    maj = math.fsum(1.0 for k in ks if k > n / 2) / m
    return ph, math.fsum(phis) / len(phis), math.fsum(qs) / len(qs), \
        math.fsum(bw) / len(bw), rho, maj


if __name__ == "__main__":
    rng = random.Random(20261113)
    print("600 items x 5 models, true mean accuracy 0.75")
    print("  true rho   p-hat    phi      Q     both-wrong (1-p)^2")
    res = {}
    for rho in (0.0, 0.25):
        ph, phi, q, bw, rh, maj = stats(table(rng, 600, 5, 0.75, rho))
        res[rho] = (ph, rh, maj)
        print("  {:<8} {:.4f}  {:.4f}  {:.4f}   {:.4f}    {:.4f}".format(
            rho, ph, phi, q, bw, (1 - ph) ** 2))
    print("moment estimate of rho and the majority it predicts")
    print("  true rho  rho-hat  predicted maj  observed maj")
    for rho in (0.0, 0.25):
        ph, rh, maj = res[rho]
        pred = bb_major(5, ph, max(rh, 0.0))
        print("  {:<8}  {:7.4f}  {:13.4f}  {:12.4f}".format(
            rho, rh, pred, maj))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《孔多塞陪审团定理与相关误差、多模型投票何时越投越准何时同源共错母典》第 3 章「怎样量出相关:两两相关、Q 统计量、等效票数与错答撞车」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-309 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026309,
  author    = {Fan, Yafu},
  title     = {Condorcet's Jury Theorem with Correlated Errors: When Multi-Model Voting Gets More Accurate and When Shared Origins Make Everyone Wrong Together},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论