Chapter 4 Projection, the Barnum Effect and Confirmation

Seeing ourselves in general statements

Evidence

In 1948 the psychologist Bertram Forer gave his students what he said was a personality description based on a test they had taken. Every student received the same text, assembled largely from horoscopes. Asked to rate how well it described them on a scale from 0 to 5, the students gave it an average of 4.26 (Forer 1949). The tendency to accept vague, general statements as specifically about oneself has since been called the Barnum effect or Forer effect.

Evidence

Lord, Ross and Lepper gave supporters and opponents of the death penalty the same pair of studies, one apparently supporting its deterrent effect and one apparently opposing it. Each side rated the study that agreed with them as more convincing and better conducted, and after reading both, the two groups were further apart than before: biased assimilation of mixed evidence produced attitude polarisation (Lord et al. 1979). Confirmation bias of this kind is one of the most widely documented patterns in human reasoning (Nickerson 1998).

A model of firm priors

Bayesian updating gives a precise language for how much evidence should move a belief. For a quantity with a Gaussian prior of mean $\mu_0$ and precision $\tau_0$ (the inverse of its variance), and observations of precision $\tau_x$ , the posterior mean is a precision-weighted average:

$\mu = \frac{\tau_0\,\mu_0 + \tau_x \sum x_i}{\tau_0 + n\,\tau_x}.$

Calculation

Three readers begin with the same belief, 2, about a quantity whose true value is 0, held with different precision, and see the same ten neutral observations.

# Precision-weighted updating of a Gaussian belief about a quantity θ.
# Prior N(μ0, 1/τ0); each observation x ~ N(θ, 1/τx). The posterior mean is
#     μ = (τ0 μ0 + τx Σx) / (τ0 + n τx)
def posterior(mu0, tau0, xs, taux=1.0):
    return (tau0 * mu0 + taux * sum(xs)) / (tau0 + taux * len(xs))

evidence = [0.0] * 10           # ten neutral observations: the true value is 0
for label, tau0 in (("open prior (τ0 = 0.5)", 0.5), ("firm prior (τ0 = 5)", 5.0),
                    ("rigid prior (τ0 = 50)", 50.0)):
    path = [posterior(2.0, tau0, evidence[:n]) for n in (0, 1, 3, 10)]
    print(f"{label:22s} belief after 0, 1, 3, 10 neutral observations: "
          + ", ".join(f"{m:.2f}" for m in path))
# open prior (τ0 = 0.5)  belief after 0, 1, 3, 10 neutral observations: 2.00, 0.67, 0.29, 0.10
# firm prior (τ0 = 5)    belief after 0, 1, 3, 10 neutral observations: 2.00, 1.67, 1.25, 0.67
# rigid prior (τ0 = 50)  belief after 0, 1, 3, 10 neutral observations: 2.00, 1.96, 1.89, 1.67

A precise prior is not irrational in itself; it is how evidence should be weighed if the prior really does rest on a great deal of earlier evidence. The problem arises when the precision is unearned, so that new evidence barely moves a belief that was never well founded.

Reading ambiguous evidence

Firm priors explain slow updating. They do not by themselves explain why two people can read the same text and grow further apart. For that, the reading of the evidence itself has to depend on the belief.

Calculation

Forty items of evidence: ten weakly for a claim, ten weakly against, twenty ambiguous. An even-handed reader sets the ambiguous items aside; an entrenched reader counts each one for the side they started on.

# Biased assimilation: the same evidence read through an entrenched stance.
# Forty items: ten weakly support claim A, ten weakly support not-A (likelihood
# ratio 1.5 either way), twenty are ambiguous. The even-handed reader ignores the
# ambiguous items; the entrenched reader counts each one for the side they started on.
items = ["A", "notA", "ambiguous", "ambiguous"] * 10

def run(p, entrenched):
    side = "A" if p > 0.5 else "notA"
    for e in items:
        if e == "ambiguous":
            if not entrenched:
                continue
            e = side
        lr = 1.5 if e == "A" else 1 / 1.5
        p = p * lr / (p * lr + (1 - p))
    return p

for start in (0.6, 0.4):
    print(f"start {start:.2f}: even-handed reader ends at {run(start, False):.2f}, "
          f"entrenched reader ends at {run(start, True):.3f}")
print("Two entrenched readers who begin 0.2 apart end at opposite extremes on identical evidence.")
# start 0.60: even-handed reader ends at 0.60, entrenched reader ends at 1.000
# start 0.40: even-handed reader ends at 0.40, entrenched reader ends at 0.000
# Two entrenched readers who begin 0.2 apart end at opposite extremes on identical evidence.

Identical evidence leaves even-handed readers where they began, and drives entrenched readers who began only slightly apart to opposite certainties. This is the structure Lord and colleagues found in people.

Projection onto machine text

Language-model output is often a natural Barnum text: fluent, general, responsive to the framing of the question, and open to more than one reading. It is therefore a surface onto which readers can project. This works in every direction. Enthusiasts may read understanding, intention or even feeling into fluent text, a tendency noticed as early as the 1960s, when users of Joseph Weizenbaum’s simple ELIZA program confided in it as if it understood them (Weizenbaum 1976). Sceptics may read evasion or emptiness into the same kind of text. Neither reading is evidence about the text by itself.

Note

Projection is a property of readers in general, not of any one group. Pointing to it does not show that a particular criticism of a model, or a particular enthusiasm, is wrong: that has to be settled by checking the specific claims. The models of this chapter describe how beliefs can resist or misread evidence; they do not tell us who is right.

Hypothesis · not established

Not established. People’s ratings of the quality of an ambiguous AI-generated text depend more on their prior attitude to AI than on the text, and the effect is larger for texts written in a general, mirror-like style than for texts making specific, checkable claims. The hypothesis predicts that, in an experiment where participants rate the same texts labelled as written by a person or by an AI, the attitude-by-label interaction is larger for general texts than for specific ones. It would be refuted if the interaction were of similar size for both kinds of text.

References

Forer, Bertram R. 1949. “The Fallacy of Personal Validation: A Classroom Demonstration of Gullibility.” Journal of Abnormal and Social Psychology 44 (1): 118–23.

Lord, Charles G., Lee Ross, and Mark R. Lepper. 1979. “Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence.” Journal of Personality and Social Psychology 37 (11): 2098–109.

Nickerson, Raymond S. 1998. “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises.” Review of General Psychology 2 (2): 175–220.

Weizenbaum, Joseph. 1976. Computer Power and Human Reason: From Judgment to Calculation. W. H. Freeman.

Fan, Y. (2026). The Algorithmic Mirror: Self-Attention Geometry, Ashby's Requisite Variety and Cognitive Projection in Information Theory and East Asian Philosophy of Mind [算法之镜] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23723557 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).