5D Library › 订阅制单位经济学、净收入留存与人工智能推理标度律 · English
第五章 重度用户、固定价格与模型路由
在模型之间路由
如果每个查询都发给最大的模型,每个查询的成本就由最大的模型决定。大多数查询并不需要它。有两种设计利用这一点:路由器预测难度,只把难的查询发给大模型;级联先试小模型,在检查器怀疑答案时再升级 (Chen et al. 2023; Ong et al. 2024)。第十段程序在一个简单的世界里比较两者,其中一次大模型调用的成本等于 15 次小模型调用。
推导
# Chapter 5. Routing between a small and a large model. Each query has a
# difficulty u, uniform on [0, 1]. The small model answers correctly when u <
# 0.70, the large one when u < 0.95. A large-model call costs 15 times a
# small-model call.
# Router: a classifier estimates u with noise (sd 0.15) and sends the query
# to the
# large model when the estimate exceeds a threshold tau.
# Cascade: always try the small model first; a checker flags a wrong small
# answer with
# probability 0.85 (and a right one with probability 0.05), and
# flagged queries are re-run on the large model (paying for both
# calls).
# 200,000 simulated queries per setting, fixed seed. Cost is per query, small
# call = 1.
import random
def router(tau, rng, n=200000):
cost = right = 0
for _ in range(n):
u = rng.random()
est = u + rng.gauss(0, 0.15)
if est > tau:
cost += 15
right += u < 0.95
else:
cost += 1
right += u < 0.70
return cost / n, right / n
def cascade(rng, catch=0.85, false_flag=0.05, n=200000):
cost = right = 0
for _ in range(n):
u = rng.random()
ok = u < 0.70
flagged = rng.random() < (false_flag if ok else catch)
cost += 1
if flagged:
cost += 15
right += u < 0.95
else:
right += ok
return cost / n, right / n
if __name__ == "__main__":
rng = random.Random(58010)
print(
" policy cost per query accuracy cost vs"
" all-large"
)
for name, c, a in (("all small", 1.0, 0.70), ("all large", 15.0, 0.95)):
print(
" {:<24} {:6.2f} {:.3f} {:5.1%}".format(
name, c, a, c / 15
)
)
for tau in (0.5, 0.6, 0.7, 0.8):
c, a = router(tau, rng)
print(" {:<24} {:6.2f} {:.3f} {:5.1%}".format(
"router, tau = {:.1f}".format(tau), c, a, c / 15))
for catch in (0.85, 0.95):
c, a = cascade(rng, catch)
print(" {:<24} {:6.2f} {:.3f} {:5.1%}".format(
"cascade, checker {:.0%}".format(catch), c, a, c / 15))
print(
"\nReading: routing buys most of the large model's accuracy for a"
" fraction of"
)
print(
"its cost, and the price of each extra point of accuracy rises"
" steeply. A"
)
print(
"cascade's value depends on how well wrong small answers are caught."
)
# policy cost per query accuracy cost vs all-large
# all small 1.00 0.700 6.7%
# all large 15.00 0.950 100.0%
# router, tau = 0.5 7.99 0.944 53.2%
# router, tau = 0.6 6.59 0.928 43.9%
# router, tau = 0.7 5.22 0.892 34.8%
# router, tau = 0.8 3.90 0.840 26.0%
# cascade, checker 85% 5.36 0.912 35.8%
# cascade, checker 95% 5.79 0.938 38.6%
#
# Reading: routing buys most of the large model's accuracy for a fraction of
# its cost, and the price of each extra point of accuracy rises steeply. A
# cascade's value depends on how well wrong small answers are caught.路由以一部分成本买到了大模型的大部分准确率。阈值为 0.7 时,路由器以全用大模型成本的 34.8% 达到 0.892 的准确率;阈值为 0.5 时,以 53.2% 达到 0.944。每多一个百分点的准确率都比上一个更贵:从 0.892 到 0.944,成本要再增加一半以上。一个能抓住 85% 小模型错误答案的检查器,使级联以 35.8% 的成本达到 0.912;更好的检查器(95%)以 38.6% 买到 0.938。在级联中,检查器就是产品本身。
注
这个玩具假设了什么。 它给路由器有噪声但无偏的难度估计,给检查器固定的捕获率。真实的路由器在可能与明天的查询不匹配的数据上训练,而一个悄悄把某一类查询路由错的路由器,会在最不显眼的地方损失准确率。程序展示的是权衡的形状,不是它在任何产品中的大小。
固定价格对上重尾
固定月费的订阅,是一个关于「平均用户的成本远低于价格」的赌注。第十一段程序为这个赌注定价。用量服从对数正态分布,中位数为每月 2 百万 token;综合服务成本为每百万 0.60 美元;价格为 20 美元。
推导
# Chapter 5. A flat monthly price against heavy-tailed usage. Price 20 dollars
# a month. Each user's monthly tokens are lognormal with median 2 million and
# log-sd sigma; the blended serving cost is 0.60 dollars per million tokens
# (of the order of p08's 70B figures at moderate batch). 200,000 simulated
# users per row, fixed seed. Part B adds selection: users who use less get
# less value and churn more (8 % a month below the median, 2 % above); 10,000
# new users sign up each month from the same distribution. How does gross
# margin drift over 24 months?
import math
import random
PRICE, MED, COST = 20.0, 2.0, 0.60
def users(sigma, n, rng):
return [MED * math.exp(sigma * rng.gauss(0, 1)) for _ in range(n)]
def margin(costs):
return 1 - sum(costs) / (PRICE * len(costs))
if __name__ == "__main__":
rng = random.Random(58011)
print("A. One month, flat price")
print(" sigma mean cost gross users costing share of cost:")
print(
" margin > price top 1 % top 10 %"
)
for sigma in (1.0, 1.5, 2.0, 2.5):
c = sorted(COST * u for u in users(sigma, 200000, rng))
tot = sum(c)
print(
" {:4.1f} {:8.2f} {:+7.1%} {:7.2%} {:6.1%} "
" {:6.1%}".format(
sigma,
tot / len(c),
margin(c),
sum(x > PRICE for x in c) / len(c),
sum(c[-2000:]) / tot,
sum(c[-20000:]) / tot,
)
)
print("\n sigma 2.0 with remedies:")
c = [COST * u for u in users(2.0, 200000, rng)]
cap = sorted(c)[int(0.99 * len(c))]
print(
" routing (cost x 0.35, p10) gross margin {:+.1%}".format(
margin([0.35 * x for x in c])
)
)
print(
" cap usage at the 99th percentile gross margin {:+.1%}".format(
margin([min(x, cap) for x in c])
)
)
print(
" both gross margin {:+.1%}".format(
margin([min(0.35 * x, 0.35 * cap) for x in c])
)
)
print("\nB. Selection over time (sigma 1.5): light users churn faster")
base = []
for month in range(1, 25):
base += users(1.5, 10000, rng)
if month in (1, 3, 6, 12, 24):
c = [COST * u for u in base]
print(
" month {:>2}: {:>7,} users, median usage {:.2f} M tokens,"
" gross margin {:+.1%}".format(
month, len(base), sorted(base)[len(base) // 2], margin(c)
)
)
base = [u for u in base if rng.random() > (0.08 if u < MED else 0.02)]
print("\nReading: the mean, not the median, sets the"
" cost, and the mean of a heavy")
print("tail is dominated by a few users. Flat"
" pricing then selects for them: the")
print("users who leave are the cheap ones, so"
" margin erodes with no change in")
print("price, model or behaviour.")
# A. One month, flat price
# sigma mean cost gross users costing share of cost:
# margin > price top 1 % top 10 %
# 1.0 1.97 +90.1% 0.25% 9.3% 39.0%
# 1.5 3.73 +81.4% 3.10% 20.6% 58.8%
# 2.0 8.68 +56.6% 8.03% 35.7% 75.7%
# 2.5 27.22 -36.1% 13.00% 57.1% 88.8%
#
# sigma 2.0 with remedies:
# routing (cost x 0.35, p10) gross margin +84.3%
# cap usage at the 99th percentile gross margin +65.5%
# both gross margin +87.9%
#
# B. Selection over time (sigma 1.5): light users churn faster
# month 1: 10,000 users, median usage 1.99 M tokens, gross margin +81.3%
# month 3: 28,516 users, median usage 2.13 M tokens, gross margin +80.5%
# month 6: 53,255 users, median usage 2.29 M tokens, gross margin +79.9%
# month 12: 93,557 users, median usage 2.58 M tokens, gross margin +79.0%
# month 24: 150,263 users, median usage 3.02 M tokens, gross margin +77.0%
#
# Reading: the mean, not the median, sets the cost, and the mean of a heavy
# tail is dominated by a few users. Flat pricing then selects for them: the
# users who leave are the cheap ones, so margin erodes with no change in
# price, model or behaviour.A 段说明为什么中位用户是错误的参照。中位用户的成本在每一行都相同,但决定毛利的平均成本随用量的离散程度上升:对数标准差为 1.0 时为 1.97 美元,2.0 时为 8.68,2.5 时为 27.22,此时毛利率为 −36.1%。对数标准差为 2.0 时,8.03% 的用户成本高于他们付的钱,最重的 1% 用户占全部服务成本的 35.7%。这就是重尾破产:为典型用户定的价,由非典型用户来买单。
补救措施改变的是算术,不是形状。对数标准差为 2.0 时,路由(成本 × 0.35,即第十段程序中阈值 0.7 的路由器)把毛利率从 +56.6% 提高到 +84.3%;在第 99 百分位封顶用量,提高到 +65.5%;两者并用,提高到 +87.9%。
B 段加入了一个让情况随时间恶化的动态。用得少的用户从产品中得到的也少,离开得更早。当轻度用户每月流失 8%、重度用户 2% 时,24 个月里客户基数的中位用量从 1.99 百万 token 升到 3.02 百万,毛利率从 +81.3% 降到 +77.0%——价格、模型和任何人的行为都没有变化。固定价格筛选出了它负担不起的用户。
假说 · 尚未验证
作者的假说(尚未验证): 在固定价格、无用量上限的 AI 订阅产品中,控制模型与价格变化后,付费用户月用量的均值与中位数之比随用户基数的年龄上升,每用户毛利随之下降。什么结果会证明它错:基数老化时均值与中位数之比保持稳定。
小结
在小模型与大模型之间路由,以大模型三分之一到一半的成本买到其大部分准确率,最后几个百分点的准确率成本急剧上升;在级联中,检查器决定结果。面对固定价格,决定毛利的是平均用户而不是中位用户,重尾用量可以把健康的毛利率变为负数;百分之几的用户可以占三分之一以上的成本。路由与封顶能恢复毛利,但随着轻度用户离开,固定价格会持续筛选出重度用户,所以除非定价跟随用量,毛利会随时间侵蚀。
参考文献
Fan, Y. (2026). Subscription Unit Economics, Net Revenue Retention and AI Inference Scaling Laws: Cohort Survival Mixtures, Negative-Churn Differential Equations, Token Compute Margins and Metcalfe Congestion Limits [订阅制单位经济学、净收入留存与人工智能推理标度律] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090164 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).