5D Library › Retrodictive Belief Locking, Sequential Pain-Point Agitation and Markovian Upsell Automata · 中文

Chapter 2. Testing Retrodiction Fairly: Sequential Tests and Selective Recall

What a fair test needs

If a reader really can read a person’s past from birth data, that can be tested, and testing is not expensive. Three things are essential: decoys (the reader or the user chooses between the true answer and several belonging to other people), a stated alternative (what hit rate “insight” means), and a stopping rule (when to conclude). Carlson’s double-blind test of astrology was designed this way: astrologers had to pick, out of three personality profiles, the one that matched a chart (Carlson 1985).

Wald’s sequential probability ratio test (SPRT) gives the most economical stopping rule: after each trial the log-likelihood ratio is added up; crossing the upper boundary accepts “insight”, crossing the lower one accepts “no insight” (Wald 1945).

The sequential test

In each trial the reader sees a client’s birth data and must pick the client’s own life timeline out of three (the other two belong to other people). Without insight the hit rate is 1/3; the claimed insight is 0.5; the type I error is 0.05, the type II error 0.10.

Calculation

# Chapter 2. [p03] A fair retrodiction test: Wald's sequential test.
# Each trial: the reader sees a client's birth data and must pick the
# client's own life timeline out of three (two decoys from other
# people). No insight: hit rate 1/3 (H0). Claimed insight: hit rate 0.5
# (H1). Wald's SPRT with alpha = 0.05, beta = 0.10 stops when the
# log-likelihood ratio leaves (log(beta/(1-alpha)), log((1-beta)/alpha)).
# 20,000 simulated test series under each hypothesis (seed 69).
import math
import random

P0, P1, ALPHA, BETA = 1 / 3, 0.5, 0.05, 0.10
UP = math.log((1 - BETA) / ALPHA)
LOW = math.log(BETA / (1 - ALPHA))
HIT = math.log(P1 / P0)
MISS = math.log((1 - P1) / (1 - P0))

def series(p, rng, cap=500):
    s, n = 0.0, 0
    while LOW < s < UP and n < cap:
        s += HIT if rng.random() < p else MISS
        n += 1
    return s >= UP, n

if __name__ == "__main__":
    print("   boundaries {:.3f} and {:.3f}; a hit adds {:.3f}, a miss {:.3f}"
          .format(LOW, UP, HIT, MISS))
    rng = random.Random(69)
    for name, p in (("H0 (no insight)", P0), ("H1 (insight 0.5)", P1)):
        runs = [series(p, rng) for _ in range(20000)]
        acc = sum(a for a, n in runs) / len(runs)
        asn = sum(n for a, n in runs) / len(runs)
        print("   {:17s}: accepts insight {:.4f}, mean trials {:.1f}".format(
            name, acc, asn))
    print("   hits in a row that alone reach the upper boundary: {}".format(
        math.ceil(UP / HIT)))
    print("\nReading: a fair test needs decoys, a stated alternative and")
    print("a stopping rule. Eight straight hits out of three-way choices")
    print("would suffice; scattered impressions never do.")
#    boundaries -2.251 and 2.890; a hit adds 0.405, a miss -0.288
#    H0 (no insight)  : accepts insight 0.0423, mean trials 38.1
#    H1 (insight 0.5) : accepts insight 0.9091, mean trials 43.8
#    hits in a row that alone reach the upper boundary: 8
#
# Reading: a fair test needs decoys, a stated alternative and
# a stopping rule. Eight straight hits out of three-way choices
# would suffice; scattered impressions never do.

The upper boundary is 2.890 and the lower -2.251; a hit adds 0.405, a miss subtracts 0.288. In the simulation a reader without insight is wrongly accepted in 0.0423 of test series, and a reader with a true hit rate of 0.5 is correctly accepted in 0.9091; the mean number of trials is about forty. Eight hits in a row alone cross the upper boundary. Real insight, in other words, would show itself within a few dozen three-way trials; scattered impressions of “being read correctly” never cross any boundary, because they have neither decoys nor a record of misses.

Selective recall

Everyday readings have no decoys and no record; users judge accuracy from memory. Let a reader without insight make a “good year” or “bad year” statement for each of the past 15 years, each right half the time; the user later recalls a right statement with probability 0.9 and a wrong one with probability 0.3.

Calculation

# Chapter 2. [p04] Selective recall turns chance into "accuracy".
# A reader with no insight makes 15 yes/no retrodictions, one for each
# of the past 15 years ("a hard year", "a good year"); each is right
# with probability 0.5. The client later recalls a right one with
# probability 0.9 and a wrong one with probability 0.3, and judges the
# reader by the share right among those she recalls. 50,000 clients
# (seed 69). Compared: perceived accuracy with recall, and accuracy
# when all 15 statements were written down beforehand.
import random

N, YEARS = 50000, 15

if __name__ == "__main__":
    rng = random.Random(69)
    seen, written, over = [], [], 0
    for _ in range(N):
        right = [rng.random() < 0.5 for _ in range(YEARS)]
        rec = [r for r in right if rng.random() < (0.9 if r else 0.3)]
        acc = sum(rec) / len(rec) if rec else 0.0
        seen.append(acc)
        written.append(sum(right) / YEARS)
        over += acc >= 0.8
    seen.sort()
    written.sort()
    print("   accuracy        mean    median  share >= 0.8")
    print("   recalled      {:.4f} {:9.4f} {:13.4f}".format(
        sum(seen) / N, seen[N // 2], over / N))
    print("   written down  {:.4f} {:9.4f} {:13.4f}".format(
        sum(written) / N, written[N // 2],
        sum(w >= 0.8 for w in written) / N))
    print("   expected recalled accuracy 0.9/(0.9+0.3) = {:.4f}".format(
        0.9 / 1.2))
    print("\nReading: with ordinary selective recall a reader with no")
    print("insight is remembered as right three times in four, and two")
    print("clients in five remember 80% or more. Writing the statements")
    print("down beforehand removes the effect entirely.")
#    accuracy        mean    median  share >= 0.8
#    recalled      0.7493    0.7500        0.4193
#    written down  0.4995    0.4667        0.0174
#    expected recalled accuracy 0.9/(0.9+0.3) = 0.7500
#
# Reading: with ordinary selective recall a reader with no
# insight is remembered as right three times in four, and two
# clients in five remember 80% or more. Writing the statements
# down beforehand removes the effect entirely.
Distribution of accuracy as recalled by users and as recorded beforehand.

The mean recalled accuracy is 0.7493, exactly 0.9/(0.9+0.3)0.9/(0.9+0.3); 0.4193 of users remember being right 80% of the time or more. If the statements are written down beforehand, mean accuracy is 0.4995, and only 0.0174 of records are right 80% of the time or more. Selective recall needs no one to cheat; it needs only that hits are easier to remember.

Hypothesis · not established

The author’s interpretation (not yet tested): a reading service that wants its users’ trust to rest on evidence should write checkable retrodictive statements down before the reading, give them to the user to keep, and check them one by one afterwards, rather than rely on the user’s memory. What would show this unnecessary: if, among real users, accuracy recorded beforehand and accuracy as recalled do not differ significantly, this chapter’s recall model overstates selective recall.

Summary

A fair test of retrodiction needs decoys, a stated alternative and a stopping rule; a sequential three-way test separates a hit rate of 0.5 from chance in about forty trials on average. Everyday “accuracy” rests on memory: statements right half the time become 0.7493 accurate when recalled and fall back to 0.4995 when recorded beforehand.

References

Carlson, Shawn. 1985. “A Double-Blind Test of Astrology.” Nature 318: 419–25.
Wald, Abraham. 1945. “Sequential Tests of Statistical Hypotheses.” Annals of Mathematical Statistics 16 (2): 117–86.

Fan, Y. (2026). Retrodictive Belief Locking, Sequential Pain-Point Agitation and Markovian Upsell Automata: Bayesian Trust Avalanches, Sunk-Cost Commitment Ratchets and Cross-Domain Transition Matrices [回溯信念锁定、序贯痛点唤醒与马尔可夫升单自动机] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090186 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).