5D Library › 五维灵商量化架构、先验条件化生成与检索增强认知体 · English

第二章 信度、档案的「形状」与盲选测验

一个维度要几道题

量表的信度通常用 Cronbach 的 α\alpha 表示 (Cronbach 1951)。当每道题与维度的关系相同(本书的载荷 0.7,同维度两题的相关为 0.49)时,kk 道题的 α\alpha 等于 Spearman-Brown 公式 kr/(1+(k−1)r)k r/(1 + (k-1)r)。

一份个人档案还有一个更容易被忽略的问题:报告给用户看的往往不是五个分数本身,而是它们的「形状」,即哪一维高、哪一维低。两个维度分数之差的信度为 (α−r12)/(1−r12)(\alpha – r_{12})/(1 – r_{12}),其中 r12r_{12} 是两个观测分数的相关。维度越相关,差值里的真实成分越少。

推导

# Chapter 2. p03: Reliability: alpha, Spearman-Brown and profile shape.
# Items load 0.7 on their dimension (p01), so two items of the same
# dimension correlate r = 0.49. For k items, Cronbach's alpha equals
# the Spearman-Brown value k r / (1 + (k - 1) r). The program gives
# alpha for k = 2, 4, 8, 10 and the k needed for 0.80 and 0.90. The
# reliability of a difference between two dimension scores (the
# "shape" of a profile) is (alpha - r12) / (1 - r12), where r12 is the
# correlation of the two observed scores, alpha phi. Finally, alpha is
# estimated from 400 simulated samples of 60 and of 300 respondents
# (4 items, phi = 0.3) to show its sampling spread (5th-95th centile).
import math
import random

R = 0.49

def sb(k, r=R):
    return k * r / (1 + (k - 1) * r)

def k_for(target, r=R):
    return math.ceil(target * (1 - r) / (r * (1 - target)))

def sample_alpha(rng, n, k=4, phi=0.3):
    rows = []
    for _ in range(n):
        g = rng.gauss(0, 1)
        f = math.sqrt(phi) * g + math.sqrt(1 - phi) * rng.gauss(0, 1)
        rows.append([0.7 * f + math.sqrt(1 - R) * rng.gauss(0, 1)
                     for _ in range(k)])
    def var(x):
        m = sum(x) / len(x)
        return sum((v - m) ** 2 for v in x) / (len(x) - 1)
    items = sum(var([r[j] for r in rows]) for j in range(k))
    total = var([sum(r) for r in rows])
    return k / (k - 1) * (1 - items / total)

if __name__ == "__main__":
    print("   items per dimension: " + "  ".join(
        "k={}: {:.4f}".format(k, sb(k)) for k in (2, 4, 8, 10)))
    print("   k needed for alpha 0.80: {}, for 0.90: {}".format(
        k_for(0.80), k_for(0.90)))
    a4 = sb(4)
    print("   reliability of a difference of two scores (k = 4):")
    for phi in (0.0, 0.3, 0.6, 0.9):
        r12 = a4 * phi
        print("     phi {:.1f}: {:.4f}".format(phi, (a4 - r12) / (1 - r12)))
    rng = random.Random(76003)
    for n in (60, 300):
        est = sorted(sample_alpha(rng, n) for _ in range(400))
        print("   alpha from {:3d} respondents: 5th {:.3f}, 95th {:.3f}"
              .format(n, est[20], est[379]))
    print("\nReading: four items give each dimension an alpha near 0.79;")
    print("0.90 needs ten. The shape of a profile is less reliable than")
    print("its parts, and much less when the dimensions correlate. A")
    print("pilot of 60 people cannot pin alpha down to better than about")
    print("a tenth.")
#    items per dimension: k=2: 0.6577  k=4: 0.7935  k=8: 0.8849  k=10: 0.9057
#    k needed for alpha 0.80: 5, for 0.90: 10
#    reliability of a difference of two scores (k = 4):
#      phi 0.0: 0.7935
#      phi 0.3: 0.7290
#      phi 0.6: 0.6059
#      phi 0.9: 0.2776
#    alpha from  60 respondents: 5th 0.701, 95th 0.855
#    alpha from 300 respondents: 5th 0.764, 95th 0.823
#
# Reading: four items give each dimension an alpha near 0.79;
# 0.90 needs ten. The shape of a profile is less reliable than
# its parts, and much less when the dimensions correlate. A
# pilot of 60 people cannot pin alpha down to better than about
# a tenth.

每维 4 道题时 α\alpha 为 0.7935,2 道题时只有 0.6577;要达到 0.80 需要 5 道,达到 0.90 需要 10 道。档案的形状更不可靠:维度相关为 0.3 时,两维之差的信度是 0.7290;相关为 0.6 时降到 0.6059;相关为 0.9 时只剩 0.2776,这时「你的 F 高于 G」几乎全是噪声。最后,用 60 人的预试估计 α\alpha,5% 到 95% 的区间是 0.701 到 0.855;300 人时收窄到 0.764 到 0.823。

报告真的描述了它的主人吗

即使分数可靠,「用户觉得报告很准」也不能证明报告有效。Forer 给学生做了一个人格测验,然后发给每人同一份笼统的描述,学生们普遍认为它准确地描述了自己 (Forer 1949)。这就是巴纳姆效应:泛泛之词人人觉得准。效度要靠区分来证明:同一种特质用不同方法测出的结果应当相近,不同特质的结果应当可以分开 (Campbell and Fiske 1959)。

一个简单的区分测验是盲选:给每位用户两份匿名报告,一份是自己的,一份是陌生人的,请他选出更符合自己的一份。若报告只是泛泛之词,命中率是 0.5。程序设用户的自我认识与真实维度的相关为 ρ\rho(0.5 或 0.3),报告的信度取上一节的三个值,比较只报告总分与报告五个分数两种情形,并计算区分命中率与 0.5 所需的人数。

推导

# Chapter 2. p04: Does a profile describe its owner? A blind-pick test.
# A rating of "how well does this fit you?" cannot separate a real
# profile from a generic one (the Barnum effect). A forced choice can:
# each user sees two anonymous reports - her own and a stranger's - and
# picks the one that fits. Model: true dimensions t ~ N(0, Phi) with
# phi = 0.3; the user's self-view v_k = rho t_k + noise (rho = 0.5 or
# 0.3); a report shows m_k = sqrt(a) t_k + sqrt(1 - a) noise, where a is
# the reliability (p03). The user picks the report closer to v. With
# 20,000 simulated pairs per cell (fixed seed) the program reports the
# hit rate when reports show only a total score or all five scores; a
# generic report hits 0.5 by construction. The last column gives users
# needed to tell the hit rate from 0.5 (two-sided 0.05, power 0.90).
import math
import random
from statistics import NormalDist

def draw_t(rng, phi=0.3):
    g = rng.gauss(0, 1)
    return [math.sqrt(phi) * g + math.sqrt(1 - phi) * rng.gauss(0, 1)
            for _ in range(5)]

def hit_rate(rng, a, rho, dims, n=20000):
    hits = 0
    for _ in range(n):
        t, u = draw_t(rng), draw_t(rng)
        v = [rho * x + math.sqrt(1 - rho * rho) * rng.gauss(0, 1) for x in t]
        rep = lambda s: [math.sqrt(a) * x + math.sqrt(1 - a) * rng.gauss(0, 1)
                         for x in s]
        mo, ms = rep(t), rep(u)
        if dims == 1:
            mo, ms, vv = [sum(mo)], [sum(ms)], [sum(v)]
        else:
            vv = v
        d_own = sum((x - y) ** 2 for x, y in zip(mo, vv))
        d_str = sum((x - y) ** 2 for x, y in zip(ms, vv))
        hits += d_own < d_str
    return hits / n

def n_needed(p, alpha=0.05, power=0.90):
    z = NormalDist().inv_cdf
    a = z(1 - alpha / 2) * 0.5
    b = z(power) * math.sqrt(p * (1 - p))
    return math.ceil(((a + b) / (p - 0.5)) ** 2)

if __name__ == "__main__":
    rng = random.Random(76004)
    print("   rho  reliability  dims  hit rate  users needed")
    for rho in (0.5, 0.3):
        for a in (0.6577, 0.7935, 0.9057):
            for dims in (1, 5):
                h = hit_rate(rng, a, rho, dims)
                print("   {:.1f} {:12.4f} {:5d} {:9.4f} {:13,d}".format(
                    rho, a, dims, h, n_needed(h)))
    print("\nReading: even a reliable profile is recognised only")
    print("modestly, because people know themselves imperfectly; five")
    print("scores are recognised more often than a total. A few dozen")
    print("to a few hundred users suffice to show it, or to show that")
    print("a report does no better than a generic one.")
#    rho  reliability  dims  hit rate  users needed
#    0.5       0.6577     1    0.6354           140
#    0.5       0.6577     5    0.7094            56
#    0.5       0.7935     1    0.6444           122
#    0.5       0.7935     5    0.7386            42
#    0.5       0.9057     1    0.6505           112
#    0.5       0.9057     5    0.7587            35
#    0.3       0.6577     1    0.5726           495
#    0.3       0.6577     5    0.6273           158
#    0.3       0.7935     1    0.5830           377
#    0.3       0.7935     5    0.6432           124
#    0.3       0.9057     1    0.5810           397
#    0.3       0.9057     5    0.6472           118
#
# Reading: even a reliable profile is recognised only
# modestly, because people know themselves imperfectly; five
# scores are recognised more often than a total. A few dozen
# to a few hundred users suffice to show it, or to show that
# a report does no better than a generic one.

即使报告可靠,用户也只能有限地认出自己的报告,因为人对自己的认识本身就不完全。ρ=0.5\rho = 0.5、信度 0.9057 时,五维报告的命中率是 0.7587,只报总分时是 0.6505;ρ=0.3\rho = 0.3 时分别是 0.6472 与 0.5810。五维报告总是比总分更容易被认出。要证明命中率高于 0.5,最有利的情形只需 35 人,最不利的需要 495 人。这是一个廉价而有力的检验:它能证明报告确实携带关于本人的信息,也能证明报告与一份泛泛之词没有区别。

假说 · 尚未验证

作者的诠释(尚未验证): 五维灵商报告在盲选测验中的命中率显著高于 0.5,且高于只报告总分的版本。什么会证明这一判断错:若在预先登记的盲选测验中,命中率与 0.5 没有显著差别,那么报告的「准确感」来自巴纳姆效应,而不是测量。

小结

每维 4 题的 α\alpha 约为 0.79,达到 0.90 需要 10 题。档案的形状比单个分数更不可靠,维度相关为 0.9 时两维之差的信度只有 0.2776。用户评价「准不准」不能证明效度;盲选测验可以,最多只需几百人。

参考文献

Campbell, Donald T., and Donald W. Fiske. 1959. “Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix.” Psychological Bulletin 56 (2): 81–105.
Cronbach, Lee J. 1951. “Coefficient Alpha and the Internal Structure of Tests.” Psychometrika 16 (3): 297–334.
Forer, Bertram R. 1949. “The Fallacy of Personal Validation: A Classroom Demonstration of Gullibility.” Journal of Abnormal and Social Psychology 44 (1): 118–23.

Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents: Multidimensional State Vectors, Personalized Prompt Conditioning and Cross-Corpus Synthesis [五维灵商量化架构、先验条件化生成与检索增强认知体] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).