关于「一个 Token 的成本:预填充、解码与内存墙」,《订阅制与 AI 的单位经济学》给出的定量结论是什么?
结论:第四章从第一性原理计算 token 成本:批量为 1 时 8B 模型的输出为每百万 3.012 美元、输入 0.018;批处理降到 0.072;长上下文拿走批处理;智能体循环重发历史,一百轮对话中输出只占成本的 6.1%,缓存把 0.6137 降到 0.0708。
数据口径:本书不是投资或财务建议;全部数据为程序生成的合成数据。 第七章与《史记·货殖列传》「贪贾三之,廉贾五之」与计然「无息币」并读,把利润率与周转率的读法明确标为诠释。
对照数据表
下表数字由本章 Python 程序原样打印,可重新运行复现。 与结论吻合的打印值: 0.018, 0.072, 3.012
| 8B: KV cache 131,072 bytes per token; prefill 0.018 dollars per | |
| million input tokens | |
| batch context decode $/M memory used KV share of | |
| output tokens bytes read | |
| 1 | 2000 3.012 20.3% 1.6% |
| 8 | 2000 0.419 22.6% 11.6% |
| 32 | 2000 0.141 30.5% 34.4% |
| 128 | 2000 0.072 61.9% 67.7% |
| 1 | 32000 3.740 25.2% 20.8% |
| 8 | 32000 1.147 61.9% 67.7% |
| 32 | 32000 does not fit in memory |
| 128 | 32000 does not fit in memory |
| 70B: KV cache 327,680 bytes per token; prefill 0.156 dollars per | |
| million input tokens | |
| batch context decode $/M memory used KV share of | |
| output tokens bytes read | |
| 1 | 2000 26.047 44.0% 0.5% |
| 8 | 2000 3.362 45.4% 3.6% |
| 32 | 2000 0.932 50.3% 13.0% |
| 128 | 2000 0.324 70.0% 37.5% |
| 1 | 32000 27.868 47.0% 7.0% |
| 8 | 32000 5.183 70.0% 37.5% |
| 32 | 32000 does not fit in memory |
| 128 | 32000 does not fit in memory |
| Reading: output tokens cost far more than input tokens at small batches, | |
| because every decode step re-reads the weights. Batching spreads that read | |
| over many users until the KV cache - which grows with context - takes over | |
| the memory and the bandwidth. Long contexts cap the batch, and so the price. |
查看完整验算程序
# Chapter 4. What a token costs to serve. An illustrative accelerator: 1.0e15
# dense FLOP/s, 3.0e12 bytes/s of memory bandwidth, 80e9 bytes of memory, 2.00
# dollars an hour. Two illustrative dense transformer sizes with grouped-query
# attention, weights in 16-bit (2 bytes per parameter):
# "8B": 8e9 parameters, 32 layers, 8 KV heads of dimension 128, on 1 device
# "70B": 70e9 parameters, 80 layers, 8 KV heads of dimension 128, on 4
# devices
# KV cache per token = 2 (K and V) x layers x KV heads x head dim x 2 bytes.
# PREFILL (reading the prompt) is compute-bound: about 2 x params FLOPs per
# token, at 50 % utilisation. DECODE (writing the answer) runs one step per
# output token for the whole batch; each step must read all weights plus every
# sequence's KV cache from memory, and also do 2 x params FLOPs per sequence;
# the step takes whichever is longer.
def model(name):
if name == "8B":
return dict(P=8e9, L=32, H=8, Dh=128, dev=1)
return dict(P=70e9, L=80, H=8, Dh=128, dev=4)
FLOPS, BW, MEM, PRICE = 1.0e15, 3.0e12, 80e9, 2.00
def kv_bytes(m):
return 2 * m["L"] * m["H"] * m["Dh"] * 2
def prefill_cost_per_m(m):
sec = 2 * m["P"] * 1e6 / (FLOPS * m["dev"] * 0.5)
return sec * PRICE * m["dev"] / 3600
def decode(m, batch, ctx):
w = 2 * m["P"]
kv = batch * ctx * kv_bytes(m)
if w + kv > MEM * m["dev"]:
return None
t_mem = (w + kv) / (BW * m["dev"])
t_flop = 2 * m["P"] * batch / (FLOPS * m["dev"])
t = max(t_mem, t_flop)
return (
t / batch * 1e6 * PRICE * m["dev"] / 3600,
(w + kv) / (MEM * m["dev"]),
kv / (w + kv),
)
if __name__ == "__main__":
for name in ("8B", "70B"):
m = model(name)
print("{}: KV cache {:,} bytes per token; prefill {:.3f} dollars per"
.format(name, kv_bytes(m), prefill_cost_per_m(m)))
print(" million input tokens")
print(
" batch context decode $/M memory used KV share of"
)
print(" output tokens bytes read")
for ctx in (2000, 32000):
for batch in (1, 8, 32, 128):
r = decode(m, batch, ctx)
if r is None:
print(
" {:>5} {:>6} does not fit in memory".format(
batch, ctx
)
)
else:
print(
" {:>5} {:>6} {:9.3f} {:6.1%} "
" {:6.1%}".format(batch, ctx, r[0], r[1], r[2])
)
print()
print(
"Reading: output tokens cost far more than input tokens at small"
" batches,"
)
print(
"because every decode step re-reads the weights. Batching spreads"
" that read"
)
print(
"over many users until the KV cache - which grows with context -"
" takes over"
)
print(
"the memory and the bandwidth. Long contexts cap the batch, and so"
" the price."
)唯一数据源与引用
范雅福(Fan, Y.). (2026). 《订阅制单位经济学、净收入留存与人工智能推理标度律》第 4 章「一个 Token 的成本:预填充、解码与内存墙」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090164
DOI: 10.5281/zenodo.24090164 · 阅读本章所在专著 · 5DL-B-058 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026058,
author = {Fan, Yafu},
title = {Subscription Unit Economics, Net Revenue Retention and AI Inference Scaling Laws},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.24090164},
url = {https://doi.org/10.5281/zenodo.24090164}
}