5D Library › Persuasive Copywriting Algebra, Loss-Framing Asymmetry and Checkout Micro-Architecture · 中文

Chapter 8. The Winner’s Curse, the Barnum Effect on Free Report Pages and a Pre-Registered Experiment

How many “winning” copy changes are real?

Calculation

# Chapter 8. p14: How many "winning" copy tests are real?
# True relative lifts of copy variants are drawn from a prior: 70% have no
# effect, 30% have a lift ~ Normal(+5%, 3%). Each test has 13,914 visitors
# per arm at a 3% baseline (p08's size for 3.0% -> 3.6%). We count the
# share of significant winners that are real, and how much the observed
# lift of a winner overstates its true lift (the winner's curse).
import random
from math import sqrt

def run(tests=4000, n=13914, base=0.03, seed=77014):
    rng = random.Random(seed)
    wins = real = 0
    over = []
    se = sqrt(2 * base * (1 - base) / n)
    for _ in range(tests):
        lift = 0.0 if rng.random() < 0.7 else rng.gauss(0.05, 0.03)
        diff = base * lift + rng.gauss(0, se)
        if diff / se > 1.959964:
            wins += 1
            real += lift > 0
            over.append(diff / base - lift)
    return wins, real, sum(over) / len(over)

if __name__ == "__main__":
    w, r, o = run()
    print("   tests 4000; significant winners {}; truly positive {} ({:.3f})"
          .format(w, r, r / w))
    print("   a winner's observed lift exceeds its true lift by {:+.1%}"
          " on average".format(o))
#    tests 4000; significant winners 210; truly positive 131 (0.624)
#    a winner's observed lift exceeds its true lift by +12.4% on average

The program assumes that seven in ten copy changes have no real effect and three in ten have a relative lift averaging 5%, with 13,914 visitors per arm in each test (Chapter 4’s sample size for 3.0% against 3.6%). Of 4,000 tests, 210 produce significant “winners”, of which 131 are truly positive, 0.624: nearly four winners in ten are false positives. Moreover the observed lift of a winner exceeds its true lift by 12.4 points on average; this is the winner’s curse: those selected are precisely the lucky ones. When an effect “fades” after launch, the usual reason is not a changed environment but an inflated original estimate.

The free report page

Calculation

# Chapter 8. p15: The free report page and the Barnum effect.
# A report built from statements that each fit a share f of all readers
# feels personal to everyone. With 8 statements, the chance a reader
# accepts at least 6 (and so feels "this is me") is binomial in the
# acceptance probability 1 - (1 - f)^M (p13 of Vol. 071, M = 10 episodes).
from math import comb

def at_least(k, n, p):
    return sum(comb(n, i) * p ** i * (1 - p) ** (n - i)
               for i in range(k, n + 1))

if __name__ == "__main__":
    print("   f      acceptance   P(6+ of 8 accepted)")
    for f in (0.05, 0.10, 0.20):
        a = 1 - (1 - f) ** 10
        print("   {:.2f}   {:10.3f}   {:.3f}".format(f, a, at_least(6, 8, a)))
    print("   the same report shown to a stranger scores the same:")
    print("   'feels accurate' measures vagueness, not personalisation.")
#    f      acceptance   P(6+ of 8 accepted)
#    0.05        0.401   0.051
#    0.10        0.651   0.431
#    0.20        0.893   0.954
#    the same report shown to a stranger scores the same:
#    'feels accurate' measures vagueness, not personalisation.

The free report page is where the sales flow of a reading product begins. If the report is built from vague statements each fitting a tenth of situations, a reader checking against ten episodes of their life accepts each with probability 0.651, and accepts 6 or more of 8 (“this is me”) with probability 0.431; with statements fitting a fifth of situations that probability rises to 0.954. Shown to a stranger, the same report scores exactly the same (Forer 1949). “The free report feels accurate” therefore measures the vagueness of its statements, not its personalisation; to base paid conversion on it is to base it on the Barnum effect.

Note

A four-step check. First, every “loss” in the copy must answer: is this a loss that buying can avert? Second, every precise number must be checkable. Third, checkout may lose only unnecessary fields; refund and cancellation terms must stay visible. Fourth, tests compute their sample size in advance, fix the stopping rule and use the refund rate as a guardrail.

A pre-registered copy experiment

Calculation

# Chapter 8. p16: A pre-registered copy test with a refund guardrail.
# Primary outcome: paid conversion, 3.0% vs 3.6% (two-sided 0.05, power
# 0.80). Guardrail: 30-day refund rate, base 3%; the variant fails if
# refunds rise by more than 2 points (one-sided 0.05, power 0.80 to
# exclude a rise when the true rates are equal). Both computed exactly
# with normal approximations; the test runs until both are met.
from math import sqrt, ceil
from statistics import NormalDist

N = NormalDist()

def n_two(p1, p2, alpha=0.05, power=0.80):
    za, zb = N.inv_cdf(1 - alpha / 2), N.inv_cdf(power)
    p = (p1 + p2) / 2
    return ceil((za * sqrt(2 * p * (1 - p))
                 + zb * sqrt(p1 * (1 - p1) + p2 * (1 - p2))) ** 2
                / (p2 - p1) ** 2)

def n_noninf(p, margin, alpha=0.05, power=0.80):
    za, zb = N.inv_cdf(1 - alpha), N.inv_cdf(power)
    return ceil((za + zb) ** 2 * 2 * p * (1 - p) / margin ** 2)

if __name__ == "__main__":
    nv = n_two(0.03, 0.036)
    buyers = n_noninf(0.03, 0.02)
    print("   visitors per arm for the conversion test: {:,}".format(nv))
    print("   buyers per arm for the refund guardrail: {:,}".format(buyers))
    print("   visitors per arm to reach those buyers at 3%: {:,}".format(
        ceil(buyers / 0.03)))
    print("   binding constraint: {}".format(
        "conversion" if nv >= buyers / 0.03 else "guardrail"))
#    visitors per arm for the conversion test: 13,914
#    buyers per arm for the refund guardrail: 900
#    visitors per arm to reach those buyers at 3%: 30,000
#    binding constraint: guardrail

The primary outcome is paid conversion, 3.0% against 3.6%, two-sided 0.05 with power 0.80, needing 13,914 visitors per arm. The guardrail is the 30-day refund rate, baseline 3%: the new copy fails if refunds rise by more than 2 points; ruling out such a rise with power 0.80 needs 900 buyers per arm, about 30,000 visitors. What binds the size of the experiment is thus the refund guardrail, not conversion itself: a test that declares a winner on conversion alone has less than half the sample needed to tell whether it is manufacturing refunds.

Hypothesis · not established

Pre-registered predictions (not yet tested): a real-loss frame of the “offer ends” kind lifts paid conversion by no more than 10% over a gain frame, with no significant rise in refunds; a frame implying loss of luck significantly raises refunds. What would show them wrong: a real-loss frame lifting conversion significantly more than 10%, or an implied-loss frame not raising refunds.

Summary of the book

The 2.25 of loss framing is a ratio of slopes of the value function (p01); realistic conversion effects are a few to a dozen per cent. The left-digit effect can be computed and puts the best price just below a multiple of ten; the credibility of precise numbers comes from checkability. Checkout friction is a product of survival rates. Putting refunds, chargebacks and repeat purchases on one ledger, real loss frames beat implied ones. Tests need a sample size fixed in advance, a fixed stopping rule and a refund guardrail.

References

Forer, Bertram R. 1949. “The Fallacy of Personal Validation: A Classroom Demonstration of Gullibility.” The Journal of Abnormal and Social Psychology 44 (1): 118–23.

Fan, Y. (2026). Persuasive Copywriting Algebra, Loss-Framing Asymmetry and Checkout Micro-Architecture: Prospect-Theory Valuation Ratios, Numerical Specificity Credibility and Cognitive-Fluency Conversion Bounds [说服文案代数、损失框架不对称性与结账微界面工程] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090205 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).