5D Library › 五维灵商量化架构、先验条件化生成与检索增强认知体 · English
第二章 信度、档案的「形状」与盲选测验
一个维度要几道题
量表的信度通常用 Cronbach 的 表示 (Cronbach 1951)。当每道题与维度的关系相同(本书的载荷 0.7,同维度两题的相关为 0.49)时, 道题的 等于 Spearman-Brown 公式 。
一份个人档案还有一个更容易被忽略的问题:报告给用户看的往往不是五个分数本身,而是它们的「形状」,即哪一维高、哪一维低。两个维度分数之差的信度为 ,其中 是两个观测分数的相关。维度越相关,差值里的真实成分越少。
推导
# Chapter 2. p03: Reliability: alpha, Spearman-Brown and profile shape.
# Items load 0.7 on their dimension (p01), so two items of the same
# dimension correlate r = 0.49. For k items, Cronbach's alpha equals
# the Spearman-Brown value k r / (1 + (k - 1) r). The program gives
# alpha for k = 2, 4, 8, 10 and the k needed for 0.80 and 0.90. The
# reliability of a difference between two dimension scores (the
# "shape" of a profile) is (alpha - r12) / (1 - r12), where r12 is the
# correlation of the two observed scores, alpha phi. Finally, alpha is
# estimated from 400 simulated samples of 60 and of 300 respondents
# (4 items, phi = 0.3) to show its sampling spread (5th-95th centile).
import math
import random
R = 0.49
def sb(k, r=R):
return k * r / (1 + (k - 1) * r)
def k_for(target, r=R):
return math.ceil(target * (1 - r) / (r * (1 - target)))
def sample_alpha(rng, n, k=4, phi=0.3):
rows = []
for _ in range(n):
g = rng.gauss(0, 1)
f = math.sqrt(phi) * g + math.sqrt(1 - phi) * rng.gauss(0, 1)
rows.append([0.7 * f + math.sqrt(1 - R) * rng.gauss(0, 1)
for _ in range(k)])
def var(x):
m = sum(x) / len(x)
return sum((v - m) ** 2 for v in x) / (len(x) - 1)
items = sum(var([r[j] for r in rows]) for j in range(k))
total = var([sum(r) for r in rows])
return k / (k - 1) * (1 - items / total)
if __name__ == "__main__":
print(" items per dimension: " + " ".join(
"k={}: {:.4f}".format(k, sb(k)) for k in (2, 4, 8, 10)))
print(" k needed for alpha 0.80: {}, for 0.90: {}".format(
k_for(0.80), k_for(0.90)))
a4 = sb(4)
print(" reliability of a difference of two scores (k = 4):")
for phi in (0.0, 0.3, 0.6, 0.9):
r12 = a4 * phi
print(" phi {:.1f}: {:.4f}".format(phi, (a4 - r12) / (1 - r12)))
rng = random.Random(76003)
for n in (60, 300):
est = sorted(sample_alpha(rng, n) for _ in range(400))
print(" alpha from {:3d} respondents: 5th {:.3f}, 95th {:.3f}"
.format(n, est[20], est[379]))
print("\nReading: four items give each dimension an alpha near 0.79;")
print("0.90 needs ten. The shape of a profile is less reliable than")
print("its parts, and much less when the dimensions correlate. A")
print("pilot of 60 people cannot pin alpha down to better than about")
print("a tenth.")
# items per dimension: k=2: 0.6577 k=4: 0.7935 k=8: 0.8849 k=10: 0.9057
# k needed for alpha 0.80: 5, for 0.90: 10
# reliability of a difference of two scores (k = 4):
# phi 0.0: 0.7935
# phi 0.3: 0.7290
# phi 0.6: 0.6059
# phi 0.9: 0.2776
# alpha from 60 respondents: 5th 0.701, 95th 0.855
# alpha from 300 respondents: 5th 0.764, 95th 0.823
#
# Reading: four items give each dimension an alpha near 0.79;
# 0.90 needs ten. The shape of a profile is less reliable than
# its parts, and much less when the dimensions correlate. A
# pilot of 60 people cannot pin alpha down to better than about
# a tenth.每维 4 道题时 为 0.7935,2 道题时只有 0.6577;要达到 0.80 需要 5 道,达到 0.90 需要 10 道。档案的形状更不可靠:维度相关为 0.3 时,两维之差的信度是 0.7290;相关为 0.6 时降到 0.6059;相关为 0.9 时只剩 0.2776,这时「你的 F 高于 G」几乎全是噪声。最后,用 60 人的预试估计 ,5% 到 95% 的区间是 0.701 到 0.855;300 人时收窄到 0.764 到 0.823。
报告真的描述了它的主人吗
即使分数可靠,「用户觉得报告很准」也不能证明报告有效。Forer 给学生做了一个人格测验,然后发给每人同一份笼统的描述,学生们普遍认为它准确地描述了自己 (Forer 1949)。这就是巴纳姆效应:泛泛之词人人觉得准。效度要靠区分来证明:同一种特质用不同方法测出的结果应当相近,不同特质的结果应当可以分开 (Campbell and Fiske 1959)。
一个简单的区分测验是盲选:给每位用户两份匿名报告,一份是自己的,一份是陌生人的,请他选出更符合自己的一份。若报告只是泛泛之词,命中率是 0.5。程序设用户的自我认识与真实维度的相关为 (0.5 或 0.3),报告的信度取上一节的三个值,比较只报告总分与报告五个分数两种情形,并计算区分命中率与 0.5 所需的人数。
推导
# Chapter 2. p04: Does a profile describe its owner? A blind-pick test.
# A rating of "how well does this fit you?" cannot separate a real
# profile from a generic one (the Barnum effect). A forced choice can:
# each user sees two anonymous reports - her own and a stranger's - and
# picks the one that fits. Model: true dimensions t ~ N(0, Phi) with
# phi = 0.3; the user's self-view v_k = rho t_k + noise (rho = 0.5 or
# 0.3); a report shows m_k = sqrt(a) t_k + sqrt(1 - a) noise, where a is
# the reliability (p03). The user picks the report closer to v. With
# 20,000 simulated pairs per cell (fixed seed) the program reports the
# hit rate when reports show only a total score or all five scores; a
# generic report hits 0.5 by construction. The last column gives users
# needed to tell the hit rate from 0.5 (two-sided 0.05, power 0.90).
import math
import random
from statistics import NormalDist
def draw_t(rng, phi=0.3):
g = rng.gauss(0, 1)
return [math.sqrt(phi) * g + math.sqrt(1 - phi) * rng.gauss(0, 1)
for _ in range(5)]
def hit_rate(rng, a, rho, dims, n=20000):
hits = 0
for _ in range(n):
t, u = draw_t(rng), draw_t(rng)
v = [rho * x + math.sqrt(1 - rho * rho) * rng.gauss(0, 1) for x in t]
rep = lambda s: [math.sqrt(a) * x + math.sqrt(1 - a) * rng.gauss(0, 1)
for x in s]
mo, ms = rep(t), rep(u)
if dims == 1:
mo, ms, vv = [sum(mo)], [sum(ms)], [sum(v)]
else:
vv = v
d_own = sum((x - y) ** 2 for x, y in zip(mo, vv))
d_str = sum((x - y) ** 2 for x, y in zip(ms, vv))
hits += d_own < d_str
return hits / n
def n_needed(p, alpha=0.05, power=0.90):
z = NormalDist().inv_cdf
a = z(1 - alpha / 2) * 0.5
b = z(power) * math.sqrt(p * (1 - p))
return math.ceil(((a + b) / (p - 0.5)) ** 2)
if __name__ == "__main__":
rng = random.Random(76004)
print(" rho reliability dims hit rate users needed")
for rho in (0.5, 0.3):
for a in (0.6577, 0.7935, 0.9057):
for dims in (1, 5):
h = hit_rate(rng, a, rho, dims)
print(" {:.1f} {:12.4f} {:5d} {:9.4f} {:13,d}".format(
rho, a, dims, h, n_needed(h)))
print("\nReading: even a reliable profile is recognised only")
print("modestly, because people know themselves imperfectly; five")
print("scores are recognised more often than a total. A few dozen")
print("to a few hundred users suffice to show it, or to show that")
print("a report does no better than a generic one.")
# rho reliability dims hit rate users needed
# 0.5 0.6577 1 0.6354 140
# 0.5 0.6577 5 0.7094 56
# 0.5 0.7935 1 0.6444 122
# 0.5 0.7935 5 0.7386 42
# 0.5 0.9057 1 0.6505 112
# 0.5 0.9057 5 0.7587 35
# 0.3 0.6577 1 0.5726 495
# 0.3 0.6577 5 0.6273 158
# 0.3 0.7935 1 0.5830 377
# 0.3 0.7935 5 0.6432 124
# 0.3 0.9057 1 0.5810 397
# 0.3 0.9057 5 0.6472 118
#
# Reading: even a reliable profile is recognised only
# modestly, because people know themselves imperfectly; five
# scores are recognised more often than a total. A few dozen
# to a few hundred users suffice to show it, or to show that
# a report does no better than a generic one.即使报告可靠,用户也只能有限地认出自己的报告,因为人对自己的认识本身就不完全。、信度 0.9057 时,五维报告的命中率是 0.7587,只报总分时是 0.6505; 时分别是 0.6472 与 0.5810。五维报告总是比总分更容易被认出。要证明命中率高于 0.5,最有利的情形只需 35 人,最不利的需要 495 人。这是一个廉价而有力的检验:它能证明报告确实携带关于本人的信息,也能证明报告与一份泛泛之词没有区别。
假说 · 尚未验证
作者的诠释(尚未验证): 五维灵商报告在盲选测验中的命中率显著高于 0.5,且高于只报告总分的版本。什么会证明这一判断错:若在预先登记的盲选测验中,命中率与 0.5 没有显著差别,那么报告的「准确感」来自巴纳姆效应,而不是测量。
小结
每维 4 题的 约为 0.79,达到 0.90 需要 10 题。档案的形状比单个分数更不可靠,维度相关为 0.9 时两维之差的信度只有 0.2776。用户评价「准不准」不能证明效度;盲选测验可以,最多只需几百人。
参考文献
Fan, Y. (2026). Spiritual Quotient Quantification, Prior-Conditioned Generation and Retrieval-Augmented Cognitive Agents: Multidimensional State Vectors, Personalized Prompt Conditioning and Cross-Corpus Synthesis [五维灵商量化架构、先验条件化生成与检索增强认知体] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090203 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).