What does Paywall Recall and Upsell Ratchets conclude, in numbers, about “Testing Message Variants: Sequential Tests, Multi-Armed Bandits and the False Discovery Rate”?
Answer: The upper boundary is a log-likelihood ratio of 2.7726 and the lower one -1.5581. A one-sided fixed-sample test with the same error rates needs 1,985 messages. When the true recovery rate is 0.030, the sequential test stops after only 982 messages on average, with a median of 697 and a ninetieth percentile of 2,078, and wrongly declares an effect in 0.045 of runs, below 0.05. When the true rate is 0.040, it needs 1,262 messages on average, with a median of 1,008, and correctly declares an effect in 0.795 of runs, close to the 0.80 that was set.
Scope of the numbers: The site is treated as a paid report product, with no claim of predictive accuracy and no fear; all visitor and order data are synthetic or illustrative. Marked boxes separate published evidence, runnable Python calculations (25 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: -1.5581, 0.030, 0.040, 0.045, 0.795, 1008
| boundaries: accept H1 at log LR 2.7726, accept H0 at -1.5581 | |||||||
| fixed-sample one-sided test with the same alpha and power: n = 1,985 | |||||||
| truth H0, p = 0.030 | mean n | 982 | median | 697 | 90th pct | 2,078 | says H1 0.045 |
| truth H1, p = 0.040 | mean n | 1,262 | median | 1,008 | 90th pct | 2,478 | says H1 0.795 |
| truth between, p = 0.035 | mean n | 1,498 | median | 1,106 | 90th pct | 3,241 | says H1 0.396 |
Show the full program
# Chapter 6. p61: Wald's sequential probability ratio test for one recall
# variant against a known baseline recovery rate.
# H0: wave-1 recovery p = 0.030; H1: p = 0.040 (a third better).
# alpha = 0.05, beta = 0.20 (power 0.80); Wald's boundaries A = (1 - beta) / alpha and
# B = beta / (1 - alpha). Simulated 2,000 times under each truth (fixed
# seed), against the fixed-sample size with the same error rates.
import math
import random
from statistics import NormalDist
P0, P1, ALPHA, BETA = 0.030, 0.040, 0.05, 0.20
LA, LB = math.log((1 - BETA) / ALPHA), math.log(BETA / (1 - ALPHA))
S1, S0 = math.log(P1 / P0), math.log((1 - P1) / (1 - P0))
def sprt(rng, p, cap=60000):
k = 0
for n in range(1, cap + 1):
k = k + (1 if rng.random() < p else 0)
llr = k * S1 + (n - k) * S0
if llr >= LA:
return n, 1
if llr <= LB:
return n, 0
return cap, -1
def fixed_n():
z = NormalDist()
za, zb = z.inv_cdf(1 - ALPHA), z.inv_cdf(1 - BETA)
num = za * math.sqrt(P0 * (1 - P0)) + zb * math.sqrt(P1 * (1 - P1))
return math.ceil((num / (P1 - P0)) ** 2)
if __name__ == "__main__":
rng = random.Random(20261120)
print("boundaries: accept H1 at log LR {:.4f}, accept H0 at {:.4f}".format(LA, LB))
print("fixed-sample one-sided test with the same alpha and power: n = {:,}".format(fixed_n()))
for name, p in (("truth H0, p = 0.030", P0), ("truth H1, p = 0.040", P1),
("truth between, p = 0.035", 0.035)):
runs = [sprt(rng, p) for _ in range(2000)]
ns = sorted(r[0] for r in runs)
ok = len([1 for r in runs if r[1] == 1]) / len(runs)
print("{:<26} mean n {:>6,.0f} median {:>6,} 90th pct {:>6,} says H1 {:.3f}".format(
name, math.fsum(ns) / len(ns), ns[len(ns) // 2], ns[int(0.9 * len(ns))], ok))Single source and citation
Fan, Y. (2026). Paywall Abandonment Recovery Timing, Personalised Curiosity Recall and High-Ticket Upsell Ratchets (Version 1.0.0), Chapter 6: Testing Message Variants: Sequential Tests, Multi-Armed Bandits and the False Discovery Rate. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-349 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026349,
author = {Fan, Yafu},
title = {Paywall Abandonment Recovery Timing, Personalised Curiosity Recall and High-Ticket Upsell Ratchets},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The 95 Per Cent Leak: The Cash Arithmetic of an RM 5,000 Daily Target
- Chapter 2 · The Decay of Intent: Three Recall Waves at Fifteen Minutes, Twenty-Four Hours and Seventy-Two Hours
- Chapter 3 · The Dose of the Information Gap: Curiosity Recalls Written Only from the Visitor's Own Free Result
- Chapter 4 · The Red Lines of Unsubscription and Complaint: A Stopping Rule, a Complaint Brake and the Consent Law of Three Jurisdictions
- Chapter 5 · The Upgrade Ratchet: Expected Value and Refund Risk of Crediting a First RM 19.90 Toward RM 199
- Chapter 7 · A Study of Classics East and West: The Han Feizi on the Difficulties of Persuasion and Aristotle's Rhetoric
- Chapter 8 · The Recall Scheduler and Upsell Trigger: Integration Spec, Example Runs and a Pre-Registered Refutation Plan