5D Library › Supercritical Viral Cascades, Information-Gap Hazard Rates and Multi-Tier Paywall Optimization · 中文
Chapter 8. A Four-Step Check, a Compliance Checklist and a Pre-Registered Paywall Test
Four questions before believing a growth plan
This chapter turns the book into four questions that can be put to any referral or paywall plan — the author’s own included. They are not business advice.
One: how was K measured? A K estimated on a tree structure is too high in a real network (Chapter 2), and a K counted by heads is inflated by fake accounts (Chapter 3). Ask for the K measured generation by generation from referral logs, the K after removing fake accounts, and the share of single seeds that die out (Chapter 1).
Two: is the acquisition cost complete? The compute and delivery cost of free readings, referral commissions and the auditing of fake accounts are all acquisition costs (Chapters 2 and 3). Ask for the full acquisition cost per paying user, not the advertising cost per sign-up.
Three: where does the extra money come from? For one good, no menu beats one price (Chapter 5). The extra money a multi-tier paywall takes should be traceable to extra goods sold or to bundles; the effect of a decoy tier must be measured separately (Chapter 6).
Four: what does the conversion rate measure? Share rates and “accuracy” ratings measure feeling (Chapter 4). Use a blind swap test to measure whether readings tell people apart.
Compliance checklist (to be checked for the place of business; not legal advice)
- Share to unlock: some platforms list “share before you can see the result” as forbidden induced sharing and restrict external links to “test” content (ifanr, n.d.).
- Commission levels: offer single-level referral rewards only; paying on a downline’s downline may amount to multi-level commission (State Council of the People’s Republic of China 2005).
- Advertising content: in mainland China, Article 9 of the Advertising Law forbids advertisements containing superstitious content (Standing Committee of the National People’s Congress 2015); other places may have special rules on advertising fortune-telling.
- Personalised pricing: quoting different prices for the same good to different people on the basis of personal data may breach rules on automated decision-making (Standing Committee of the National People’s Congress 2021); self-selected tiers from a public menu carry less risk.
- Repeated charging and anxiety marketing: using anxiety or fading curiosity to push users to pay again and again for the same matter may breach consumer-protection rules.
Experimenting when users recruit each other
When comparing two paywalls, randomising by visitor goes wrong: the version an invited friend sees depends on the inviter, and the two arms contaminate each other. The remedy is to randomise whole referral trees, at the price of a design effect , where is the mean tree size and the intra-tree correlation.
Calculation
# Chapter 8. Testing a paywall when users recruit each other.
# Comparing two paywall designs by randomising individual visitors fails
# when users invite friends: an invited friend's behaviour depends on the
# inviter's version (interference). The remedy is to randomise whole
# referral trees - each seed and everyone it brings in gets the seed's
# version - and to pay for it with a design effect
# DE = 1 + (mean tree size - 1) x ICC,
# ICC the intra-tree correlation of the outcome. Outcome here: whether a
# visitor pays within 30 days, 5.0 % under A; smallest effect of interest
# +1.0 point (6.0 % under B); two-sided alpha 0.05, power 80 %.
# Part 2 shows the bias of individual randomisation when a share c of
# visitors in arm A arrived through a B-arm inviter and behave like B.
import math
from statistics import NormalDist
Z = NormalDist()
def n_individual(p1, p2, alpha=0.05, power=0.8):
za, zb = Z.inv_cdf(1 - alpha / 2), Z.inv_cdf(power)
pbar = (p1 + p2) / 2
num = (za * math.sqrt(2 * pbar * (1 - pbar))
+ zb * math.sqrt(p1 * (1 - p1) + p2 * (1 - p2))) ** 2
return math.ceil(num / (p2 - p1) ** 2)
if __name__ == "__main__":
n = n_individual(0.05, 0.06)
print(" individual randomisation: {:,} visitors per arm".format(n))
print("\n mean tree size ICC design effect visitors per arm")
for size in (3, 8):
for icc in (0.02, 0.05, 0.20):
de = 1 + (size - 1) * icc
print(" {:2d} {:4.2f} {:5.2f} {:,}"
.format(size, icc, de, math.ceil(n * de)))
print("\n contaminated share c measured difference (true +1.0)")
for c in (0.0, 0.1, 0.3, 0.5):
print(" {:3.1f} {:+.2f} points".format(
c, 1.0 * (1 - c)))
print("\nReading: randomising referral trees costs a design effect that")
print("can more than double the sample; randomising individuals instead")
print("shrinks the measured effect towards zero by the share of")
print("visitors who arrived through the other arm.")
# individual randomisation: 8,158 visitors per arm
#
# mean tree size ICC design effect visitors per arm
# 3 0.02 1.04 8,485
# 3 0.05 1.10 8,974
# 3 0.20 1.40 11,422
# 8 0.02 1.14 9,301
# 8 0.05 1.35 11,014
# 8 0.20 2.40 19,580
#
# contaminated share c measured difference (true +1.0)
# 0.0 +1.00 points
# 0.1 +0.90 points
# 0.3 +0.70 points
# 0.5 +0.50 points
#
# Reading: randomising referral trees costs a design effect that
# can more than double the sample; randomising individuals instead
# shrinks the measured effect towards zero by the share of
# visitors who arrived through the other arm.With individual randomisation, seeing a rise in the pay rate from 5.0 % to 6.0 % with 80 % power needs 8,158 visitors per arm. With tree randomisation, a mean tree size of 8 and an intra-tree correlation of 0.20 give a design effect of 2.40 and need 19,580 visitors per arm. If individuals are randomised anyway and half the visitors arrive through an inviter in the other arm, the measured difference is only half the true one (+0.50 points).
A pre-registered paywall test
Hypothesis · not established
Protocol. Compare A (one price of 9.9 for the full reading) with B (the tiered menu). The unit of randomisation is the referral tree. Primary outcome: net revenue per seed over 30 days, counting every visitor in the tree: payments minus commissions, minus the cost of the free readings given (0.15 each). Primary estimand : the mean of B minus the mean of A; the smallest effect of interest is 0.10 per seed. Three gates make a result uninterpretable rather than negative: a sample-ratio mismatch in the split of trees (), more than 5 % of trees exposed to both versions, and enrolment below 90 % of plan. People independent of the pricing decision run and analyse the test with the registered code, and the results are published whether positive or negative.
Calculation
# Chapter 8. A pre-registered paywall test and its decision rule.
# Arms: A = one price for the full reading (9.9); B = the tiered menu.
# Unit of randomisation: the referral tree (p15). Primary outcome: NET
# revenue per seed visitor over 30 days, counting every visitor in the
# seed's tree: payments minus commissions paid out minus the cost of the
# free readings given (0.15 each). Estimand D = mean(B) - mean(A); the
# smallest effect of interest is 0.10 per seed. Gates that make a result
# uninterpretable rather than negative: sample-ratio mismatch (|z| > 3 on
# the split of trees), more than 5 % of trees exposed to both arms, and
# fewer than 90 % of the planned trees. The program sizes the test by
# simulating trees (outcome per seed: a heavy-tailed sum, as in p02; a
# tenth of payments paid out as commission; B lifts paying by 0.4 point)
# and prints the decision for each branch of the rule.
import math
import random
from statistics import NormalDist
Z = NormalDist()
def tree_outcome(rng, uplift):
size = 1
alive = 1
while alive and size < 400:
kids = sum(rng.random() < 0.18 for _ in range(5))
alive += kids - 1
size += kids
pays = sum(rng.random() < 0.05 + uplift for _ in range(size))
return pays * 9.9 * 0.9 - 0.15 * size
def decide(d, se, srm_z, mixed, enrolled):
if abs(srm_z) > 3:
return "uninterpretable: sample-ratio mismatch"
if mixed > 0.05:
return "uninterpretable: arms mixed in more than 5 % of trees"
if enrolled < 0.9:
return "uninterpretable: under 90 % of planned trees"
lo, hi = d - 1.96 * se, d + 1.96 * se
if lo > 0.10:
return "B better by at least the smallest effect of interest"
if hi < 0:
return "B worse: the tiered menu earns less per seed"
if lo > 0:
return "B better, but not shown to reach 0.10"
return "no difference shown"
if __name__ == "__main__":
rng = random.Random(63016)
a = [tree_outcome(rng, 0.0) for _ in range(20000)]
b = [tree_outcome(rng, 0.004) for _ in range(20000)]
ma, mb = sum(a) / len(a), sum(b) / len(b)
sd = math.sqrt((sum((x - ma) ** 2 for x in a)
+ sum((x - mb) ** 2 for x in b)) / (len(a) + len(b) - 2))
n = math.ceil(2 * ((1.96 + Z.inv_cdf(0.8)) * sd / 0.10) ** 2)
print(" simulated net revenue per seed: A {:.3f}, B {:.3f}".format(
ma, mb))
print(" pooled SD {:.3f} -> {:,} trees per arm for 80 % power"
.format(sd, n))
se = sd * math.sqrt(2 / n)
print("\n decision rule, one line per branch:")
for case in ((0.25, 0.0, 0.0, 1.0), (0.12, 0.0, 0.0, 1.0),
(0.05, 0.0, 0.0, 1.0), (-0.20, 0.0, 0.0, 1.0),
(0.25, 3.5, 0.0, 1.0), (0.25, 0.0, 0.08, 1.0),
(0.25, 0.0, 0.0, 0.8)):
print(" D = {:+.2f}: {}".format(case[0], decide(case[0], se,
*case[1:])))
print("\nReading: because one seed's tree may hold hundreds of users,")
print("the outcome is so dispersed that detecting 0.10 per seed takes")
print("well over a hundred thousand trees per arm. A test that small")
print("sites cannot afford should not be replaced by a guess.")
# simulated net revenue per seed: A 2.987, B 3.253
# pooled SD 10.308 -> 166,788 trees per arm for 80 % power
#
# decision rule, one line per branch:
# D = +0.25: B better by at least the smallest effect of interest
# D = +0.12: B better, but not shown to reach 0.10
# D = +0.05: no difference shown
# D = -0.20: B worse: the tiered menu earns less per seed
# D = +0.25: uninterpretable: sample-ratio mismatch
# D = +0.25: uninterpretable: arms mixed in more than 5 % of trees
# D = +0.25: uninterpretable: under 90 % of planned trees
#
# Reading: because one seed's tree may hold hundreds of users,
# the outcome is so dispersed that detecting 0.10 per seed takes
# well over a hundred thousand trees per arm. A test that small
# sites cannot afford should not be replaced by a guess.In the simulation the two arms’ net revenue per seed is 2.987 and 3.253, with a pooled standard deviation of 10.308 — one seed’s referral tree can hold hundreds of people, and the outcome is extremely dispersed. Seeing a difference of 0.10 with 80 % power needs 166,788 referral trees per arm. The decision rule prints one line per branch: B better by at least the smallest effect of interest; B better but not shown to reach 0.10; no difference shown; B worse; and the three uninterpretable cases.
Note
What the result would and would not mean. It describes the referral trees during the test, under these two designs. A small site that cannot afford a test of this size should not replace it with a guess; it can test for a larger effect instead, or lengthen the test with interim-analysis rules registered in advance.
Summary
Four questions sum up the book: whether K was measured generation by generation from referral logs with fake accounts removed; whether the acquisition cost counts free readings, commissions and auditing; whether the extra money comes from extra goods sold; and whether conversion and accuracy ratings measure feeling or discrimination. The compliance checklist lists five rules to check: share to unlock, commission levels, advertising content, personalised pricing and repeated charging. When users recruit each other, experiments should randomise referral trees; a pre-registered test with net revenue per seed as its primary outcome needs 166,788 trees per arm.
References
Fan, Y. (2026). Supercritical Viral Cascades, Information-Gap Hazard Rates and Multi-Tier Paywall Optimization: Galton-Watson Referral Thresholds, Share-to-Unlock Incentives and Second-Degree Price Discrimination [超临界病毒级联、信息缺口风险率与多层付费墙优化] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090174 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).