What does Copy and Checkout Arithmetic conclude, in numbers, about “Cognitive Fluency and the Statistical Limits of Testing”?
Answer: Chapter 4 measures the fluency of copy and computes A/B sample sizes (13,914 visitors per arm for 3.0% against 3.6%) and the cost of peeking (false-positive rate 0.256 with 20 looks).
Scope of the numbers: Conversion rates, refund rates and costs not attributed to a source are illustrative; all data are generated by its programs; the book is not legal advice. Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.256, 13914, 20
| baseline | target | visitors per arm |
|---|---|---|
| 0.030 | 0.033 | 53,211 |
| 0.030 | 0.036 | 13,914 |
| 0.030 | 0.045 | 2,518 |
| 0.100 | 0.110 | 14,751 |
| identical pages, | 1 looks: false-positive rate 0.044 | |
| identical pages, | 5 looks: false-positive rate 0.134 | |
| identical pages, 20 looks: false-positive rate 0.256 |
Show the full program
# Chapter 4. p08: How many visitors to see a lift - and the peeking trap.
# Two-proportion z test, two-sided alpha 0.05, power 0.80:
# n per arm = (z_a sqrt(2 p q) + z_b sqrt(p1 q1 + p2 q2))^2 / (p2 - p1)^2.
# Peeking: checking the p-value after every 500 visitors per arm and
# stopping at the first p < 0.05 inflates the false-positive rate even
# when the two pages are identical. Simulated with a fixed seed.
import random
from math import sqrt, ceil
from statistics import NormalDist
N = NormalDist()
def n_per_arm(p1, p2, alpha=0.05, power=0.80):
za, zb = N.inv_cdf(1 - alpha / 2), N.inv_cdf(power)
p = (p1 + p2) / 2
num = (za * sqrt(2 * p * (1 - p))
+ zb * sqrt(p1 * (1 - p1) + p2 * (1 - p2)))
return ceil(num ** 2 / (p2 - p1) ** 2)
def peeking_fpr(p, looks, step, trials, rng):
hits = 0
for _ in range(trials):
a = b = n = 0
for _ in range(looks):
a += sum(rng.random() < p for _ in range(step))
b += sum(rng.random() < p for _ in range(step))
n += step
pool = (a + b) / (2 * n)
se = sqrt(2 * pool * (1 - pool) / n) or 1
if abs(a - b) / n / se > 1.959964:
hits += 1
break
return hits / trials
if __name__ == "__main__":
print(" baseline target visitors per arm")
for p1, p2 in ((0.03, 0.033), (0.03, 0.036), (0.03, 0.045),
(0.10, 0.11)):
print(" {:8.3f} {:6.3f} {:,}".format(p1, p2, n_per_arm(p1, p2)))
rng = random.Random(77008)
for looks in (1, 5, 20):
print(" identical pages, {:2d} looks: false-positive rate {:.3f}"
.format(looks, peeking_fpr(0.05, looks, 500, 1000, rng)))Single source and citation
Fan, Y. (2026). Persuasive Copywriting Algebra, Loss-Framing Asymmetry and Checkout Micro-Architecture (Version 1.0.0), Chapter 4: Cognitive Fluency and the Statistical Limits of Testing. 5D Library. https://doi.org/10.5281/zenodo.24090205
DOI: 10.5281/zenodo.24090205 · Read the monograph · 5DL-B-077 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026077,
author = {Fan, Yafu},
title = {Persuasive Copywriting Algebra, Loss-Framing Asymmetry and Checkout Micro-Architecture},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.24090205},
url = {https://doi.org/10.5281/zenodo.24090205}
}Other chapter answers in this book
- Chapter 1 · The Prospect-Theory Arithmetic of Loss Frames
- Chapter 2 · Signalling Models of Price Endings and Numerical Precision
- Chapter 3 · The Friction Equations of Checkout Micro-Architecture
- Chapter 5 · The Full Ledger of a Fear Frame: Refunds, Chargebacks and Repeat Purchases
- Chapter 6 · The Ceiling of the Funnel and the Best Price
- Chapter 7 · Han Feizi, the Wenxin diaolong and the Rhetoric of Gain and Harm
- Chapter 8 · The Winner's Curse, the Barnum Effect on Free Report Pages and a Pre-Registered Experiment