5D Library › 回溯信念锁定、序贯痛点唤醒与马尔可夫升单自动机 · English

第二章 公平地检验回溯:序贯检验与选择性回忆

一个公平的检验需要什么

如果测算师确实能从出生资料中读出一个人的过去,这是可以检验的,而且检验并不昂贵。关键是三件事:诱饵(让测算师或用户在真实答案与若干别人的答案之间选择)、事先声明的备择假设(「洞察力」意味着多高的命中率),以及停止规则(什么时候下结论)。Carlson 对占星的双盲检验正是这样设计的:占星师要从三份人格测验结果中挑出与星盘相符的一份 (Carlson 1985)。

Wald 的序贯概率比检验(SPRT)给出最节省的停止规则:每做一次试验,就把对数似然比累加起来,越过上界即接受「有洞察力」,越过下界即接受「无洞察力」(Wald 1945)。

序贯检验

设每次试验,测算师看到一位客户的出生资料,要从三条人生时间线中(另两条来自别人)挑出客户自己的那一条。无洞察力时命中率为 1/3,声称的洞察力为 0.5;第一类错误 0.05,第二类错误 0.10。

推导

# Chapter 2. [p03] A fair retrodiction test: Wald's sequential test.
# Each trial: the reader sees a client's birth data and must pick the
# client's own life timeline out of three (two decoys from other
# people). No insight: hit rate 1/3 (H0). Claimed insight: hit rate 0.5
# (H1). Wald's SPRT with alpha = 0.05, beta = 0.10 stops when the
# log-likelihood ratio leaves (log(beta/(1-alpha)), log((1-beta)/alpha)).
# 20,000 simulated test series under each hypothesis (seed 69).
import math
import random

P0, P1, ALPHA, BETA = 1 / 3, 0.5, 0.05, 0.10
UP = math.log((1 - BETA) / ALPHA)
LOW = math.log(BETA / (1 - ALPHA))
HIT = math.log(P1 / P0)
MISS = math.log((1 - P1) / (1 - P0))

def series(p, rng, cap=500):
    s, n = 0.0, 0
    while LOW < s < UP and n < cap:
        s += HIT if rng.random() < p else MISS
        n += 1
    return s >= UP, n

if __name__ == "__main__":
    print("   boundaries {:.3f} and {:.3f}; a hit adds {:.3f}, a miss {:.3f}"
          .format(LOW, UP, HIT, MISS))
    rng = random.Random(69)
    for name, p in (("H0 (no insight)", P0), ("H1 (insight 0.5)", P1)):
        runs = [series(p, rng) for _ in range(20000)]
        acc = sum(a for a, n in runs) / len(runs)
        asn = sum(n for a, n in runs) / len(runs)
        print("   {:17s}: accepts insight {:.4f}, mean trials {:.1f}".format(
            name, acc, asn))
    print("   hits in a row that alone reach the upper boundary: {}".format(
        math.ceil(UP / HIT)))
    print("\nReading: a fair test needs decoys, a stated alternative and")
    print("a stopping rule. Eight straight hits out of three-way choices")
    print("would suffice; scattered impressions never do.")
#    boundaries -2.251 and 2.890; a hit adds 0.405, a miss -0.288
#    H0 (no insight)  : accepts insight 0.0423, mean trials 38.1
#    H1 (insight 0.5) : accepts insight 0.9091, mean trials 43.8
#    hits in a row that alone reach the upper boundary: 8
#
# Reading: a fair test needs decoys, a stated alternative and
# a stopping rule. Eight straight hits out of three-way choices
# would suffice; scattered impressions never do.

上界为 2.890,下界为 -2.251;一次命中加 0.405,一次失误减 0.288。模拟中,无洞察力的测算师被误判为有洞察力的比例是 0.0423,确有 0.5 命中率的测算师被正确接受的比例是 0.9091;平均试验次数约为四十。连续 8 次命中就能单独越过上界。换句话说,真实的洞察力是可以在几十次三选一的试验中显现的;零散的「说中」印象则永远越不过任何界限,因为它们没有诱饵,也没有失误的记录。

选择性回忆

日常的测算没有诱饵与记录,用户凭记忆评价准不准。设测算师没有洞察力,对过去 15 年逐年作出「好年」或「坏年」的断言,每条对错各半;用户之后记得说对的断言的概率为 0.9,记得说错的为 0.3。

推导

# Chapter 2. [p04] Selective recall turns chance into "accuracy".
# A reader with no insight makes 15 yes/no retrodictions, one for each
# of the past 15 years ("a hard year", "a good year"); each is right
# with probability 0.5. The client later recalls a right one with
# probability 0.9 and a wrong one with probability 0.3, and judges the
# reader by the share right among those she recalls. 50,000 clients
# (seed 69). Compared: perceived accuracy with recall, and accuracy
# when all 15 statements were written down beforehand.
import random

N, YEARS = 50000, 15

if __name__ == "__main__":
    rng = random.Random(69)
    seen, written, over = [], [], 0
    for _ in range(N):
        right = [rng.random() < 0.5 for _ in range(YEARS)]
        rec = [r for r in right if rng.random() < (0.9 if r else 0.3)]
        acc = sum(rec) / len(rec) if rec else 0.0
        seen.append(acc)
        written.append(sum(right) / YEARS)
        over += acc >= 0.8
    seen.sort()
    written.sort()
    print("   accuracy        mean    median  share >= 0.8")
    print("   recalled      {:.4f} {:9.4f} {:13.4f}".format(
        sum(seen) / N, seen[N // 2], over / N))
    print("   written down  {:.4f} {:9.4f} {:13.4f}".format(
        sum(written) / N, written[N // 2],
        sum(w >= 0.8 for w in written) / N))
    print("   expected recalled accuracy 0.9/(0.9+0.3) = {:.4f}".format(
        0.9 / 1.2))
    print("\nReading: with ordinary selective recall a reader with no")
    print("insight is remembered as right three times in four, and two")
    print("clients in five remember 80% or more. Writing the statements")
    print("down beforehand removes the effect entirely.")
#    accuracy        mean    median  share >= 0.8
#    recalled      0.7493    0.7500        0.4193
#    written down  0.4995    0.4667        0.0174
#    expected recalled accuracy 0.9/(0.9+0.3) = 0.7500
#
# Reading: with ordinary selective recall a reader with no
# insight is remembered as right three times in four, and two
# clients in five remember 80% or more. Writing the statements
# down beforehand removes the effect entirely.
用户回忆中的准确率与事先记录的准确率的分布。

用户回忆中的平均准确率是 0.7493,恰好等于 0.9/(0.9+0.3)0.9/(0.9+0.3);0.4193 的用户记得自己被说中了 80% 以上。若断言事先写下来,平均准确率是 0.4995,记录中说中 80% 以上的只有 0.0174。选择性回忆不需要任何人作假,它只需要人们更容易记住命中。

假说 · 尚未验证

作者的诠释(尚未验证): 一个测算服务若要让用户的信任建立在证据之上,就应当在测算之前把可以核对的回溯断言写下来、交给用户保存,事后逐条核对,而不是依赖用户的回忆。什么会证明这一做法没有必要:若事先记录的准确率与用户回忆的准确率在真实用户中没有显著差别,说明本章的回忆模型夸大了选择性回忆。

小结

公平的回溯检验需要诱饵、事先声明的备择假设与停止规则;三选一的序贯检验平均四十次试验左右就能区分 0.5 的命中率与随机。日常的「说中」依赖回忆:对错各半的断言在回忆中变成 0.7493 的准确率,事先记录则回到 0.4995。

参考文献

Carlson, Shawn. 1985. “A Double-Blind Test of Astrology.” Nature 318: 419–25.
Wald, Abraham. 1945. “Sequential Tests of Statistical Hypotheses.” Annals of Mathematical Statistics 16 (2): 117–86.

Fan, Y. (2026). Retrodictive Belief Locking, Sequential Pain-Point Agitation and Markovian Upsell Automata: Bayesian Trust Avalanches, Sunk-Cost Commitment Ratchets and Cross-Domain Transition Matrices [回溯信念锁定、序贯痛点唤醒与马尔可夫升单自动机] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090186 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).