5D Library › 订阅制单位经济学、净收入留存与人工智能推理标度律 · English

第八章 四步核查与一项预注册的价格检验

相信一个订阅模型之前的四个问题

本章把全书变成四个问题,可以向任何订阅制计划或路演——包括自己的——提出。它们不是投资建议。

一、哪一个流失率,来自哪个模型? 用单一流失率算出的终身价值是错的,错的方向取决于它所取基数的年龄(第一章)。要看队列存活曲线、折现率,以及至少两种异质性模型给出的价值区间。如果终身价值没有折现,问一问折现后是多少。

二、NRR 处在生命的哪个阶段? 高于 100% 的 NRR 把存量客户变成第二台引擎(第二章),但扩张是有界的,混合 NRR 随基数老化而下降。要看按队列年龄分的 NRR,而不只是混合值,也不要外推一家年轻公司的 NRR。

三、这个增长计划的低谷有多深? LTV / CAC 对时间只字未提(第三章)。要看获客成本回收期、折现后的比率,以及按计划自身增长路径计算的现金低谷及其月份。如果计划的资金规模是按比目标更慢的路径定的,这个计划就是一个在成功时把钱烧光的计划。

四、对 AI 产品:平均用户的成本是多少? 决定毛利的不是中位用户(第五章)。要看用量的分布、在实际服务的批量与上下文长度下每百万 token 的成本(第四章)、最重 1% 用户的成本占比,以及基数的用量如何随其年龄变化。

一项预注册的价格检验

第七章以一个只有实验才能解决的冲突结束:更高的价格提高首年收入,但如果流失对价格敏感,可能降低每位注册用户的价值。下面的检验在任何数据出现之前写成。

假说 · 尚未验证

研究方案。 新注册用户在注册时被随机分到每月 20 或 25 的价格;其他一切相同。主要结局:每位注册用户前 12 个月的毛利——收入减去服务成本,按每月 1% 折现——使转化、流失与用量成本都计算在内。主要估计量 DD:高价组均值减低价组均值;为正支持高价。最小关注效应:每位注册用户 5。不影响现有客户。两组都在付款前披露价格。检验由独立于定价决策的人员实施与分析,不是对任何价格的推荐。

推导

# Chapter 8. A pre-registered price test. New sign-ups are randomised to price
# A = 20 or price B = 25 a month. Primary outcome: gross profit per SIGN-UP
# over the first 12 months (revenue minus serving cost, discounted 1 % a
# month), so conversion, churn and usage cost all count. D = mean(B) -
# mean(A); POSITIVE D favours the higher price. Smallest effect of interest 5
# per sign-up. The standard deviation of the outcome is taken from a
# simulation of the book's own model: trial-to-paid conversion 0.30 at 20 and
# 0.27 at 25, churn c(p) from p15, serving cost per month lognormal with
# median 1.20 and log-sd 1.5 (p11). Two-sided alpha 0.05, power 0.80.
import math
import random

def signup(p, conv, rng):
    if rng.random() >= conv:
        return 0.0
    c = 0.02 * math.exp((p - 20) / 15)
    cost = 1.20 * math.exp(1.5 * rng.gauss(0, 1))
    gp = 0.0
    for t in range(12):
        gp += (p - cost) / 1.01 ** t
        if rng.random() < c:
            break
    return gp

SESOI = 5.0

def verdict(d, lo, hi, n_per_arm, planned, srm_p=0.5, price_shown_ok=True):
    if not price_shown_ok:
        return (
            "D = {:+.2f}: NOT INTERPRETABLE - some sign-ups saw the other"
            " arm's price".format(d)
        )
    if srm_p < 0.001:
        return ("D = {:+.2f}: NOT INTERPRETABLE - arm sizes differ more than"
                "\n    randomisation allows".format(d))
    if n_per_arm < 0.9 * planned:
        return (
            "D = {:+.2f}: NOT INTERPRETABLE - only {:,} of {:,} planned"
            " sign-ups per arm".format(d, n_per_arm, planned)
        )
    if lo >= SESOI:
        v = (
            "the higher price earns more per sign-up,\n    by at least the"
            " SESOI"
        )
    elif lo > 0:
        v = "the higher price earns more,\n    not shown to reach the SESOI"
    elif hi < 0:
        v = "the higher price earns LESS per sign-up"
    elif lo >= -SESOI and hi <= SESOI:
        v = "no difference worth acting on"
    else:
        v = "unclear: the interval is too wide to decide"
    return "D = {:+.2f} ({:+.2f} to {:+.2f}): {}".format(d, lo, hi, v)

if __name__ == "__main__":
    rng = random.Random(58016)
    a = [signup(20, 0.30, rng) for _ in range(100000)]
    b = [signup(25, 0.27, rng) for _ in range(100000)]
    ma, mb = sum(a) / len(a), sum(b) / len(b)
    sd = math.sqrt(
        (sum((x - ma) ** 2 for x in a) + sum((x - mb) ** 2 for x in b))
        / (len(a) + len(b) - 2)
    )
    print(
        "Model means per sign-up: A {:.2f}, B {:.2f} (true D = {:+.2f});"
        " pooled sd {:.1f}".format(ma, mb, mb - ma, sd)
    )
    n = math.ceil(2 * (1.959964 + 0.841621) ** 2 * sd ** 2 / SESOI ** 2)
    print("Sign-ups needed per arm: {:,}".format(n))
    print("\nDecision rule, fixed before the data are examined:")
    for c in (
        (9.1, 6.2, 12.0, n, n),
        (3.0, 0.4, 5.6, n, n),
        (0.8, -2.0, 3.6, n, n),
        (-6.5, -9.3, -3.7, n, n),
        (2.0, -6.0, 10.0, n, n),
        (9.1, 6.2, 12.0, n, n, 0.0001),
        (9.1, 6.2, 12.0, int(0.6 * n), n),
        (9.1, 6.2, 12.0, n, n, 0.5, False),
    ):
        print(verdict(*c))
    print("\nThe example intervals above are invented, to"
          " show what each branch prints.")
    print("Secondary outcomes (conversion, 12-month"
          " churn, serving cost per paying")
    print("user) are reported with intervals and never change the verdict.")
# Model means per sign-up: A 49.85, B 56.39 (true D = +6.54); pooled sd 112.2
# Sign-ups needed per arm: 7,908
#
# Decision rule, fixed before the data are examined:
# D = +9.10 (+6.20 to +12.00): the higher price earns more per sign-up,
#     by at least the SESOI
# D = +3.00 (+0.40 to +5.60): the higher price earns more,
#     not shown to reach the SESOI
# D = +0.80 (-2.00 to +3.60): no difference worth acting on
# D = -6.50 (-9.30 to -3.70): the higher price earns LESS per sign-up
# D = +2.00 (-6.00 to +10.00): unclear: the interval is too wide to decide
# D = +9.10: NOT INTERPRETABLE - arm sizes differ more than
#     randomisation allows
# D = +9.10: NOT INTERPRETABLE - only 4,744 of 7,908 planned sign-ups per arm
# D = +9.10: NOT INTERPRETABLE - some sign-ups saw the other arm's price
#
# The example intervals above are invented, to show what each branch prints.
# Secondary outcomes (conversion, 12-month churn, serving cost per paying
# user) are reported with intervals and never change the verdict.

本书自己的模型预测 D=+6.54D = +6.54,刚好高于最小关注效应,合并标准差为每位注册用户 112.2——大多数注册用户从不转化,少数重度用户的成本高于他们付的钱,所以结局极其分散。正是这种分散,使检验每组需要 7,908 位注册用户。

判定规则对每个分支打印一行。三道闸门使结果不可解释,而不是否定:样本比例失衡(两组人数之差超过随机化所允许的范围,这是实验出错最常见的迹象 (Kohavi et al. 2020))、部分注册用户看到了另一组的价格、以及注册人数不足计划的 90%。高价组每位注册用户赚得更少,是一个可能且有信息量的结局,规则为它命名。

注

结果意味着什么、不意味着什么。 它描述的是检验期间、在这两个价格下的注册用户。它不说明现有客户在涨价后会怎样,不说明未测试的价格会怎样,也不说明效应能否持续到十二个月之后——在那里,第一章的预测问题又回来了。

小结

四个问题概括了全书:终身价值依据的是哪个流失率、哪个模型,NRR 处于基数生命的哪个阶段,增长计划意味着多深的现金低谷,以及——对 AI 产品——在实际服务的批量与上下文下平均用户的成本是多少。本书指出的一个冲突,即首年收入与每位注册用户价值之间的冲突,可以用一项随机价格检验来解决;其主要结局把转化、流失与服务成本合在一起计算,正因为这个结局如此分散,每组需要 7,908 位注册用户。

参考文献

Kohavi, Ron, Diane Tang, and Ya Xu. 2020. Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press.

Fan, Y. (2026). Subscription Unit Economics, Net Revenue Retention and AI Inference Scaling Laws: Cohort Survival Mixtures, Negative-Churn Differential Equations, Token Compute Margins and Metcalfe Congestion Limits [订阅制单位经济学、净收入留存与人工智能推理标度律] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090164 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).