关于「一个 Token 的成本:预填充、解码与内存墙」,《订阅制与 AI 的单位经济学》给出的定量结论是什么?

结论:第四章从第一性原理计算 token 成本:批量为 1 时 8B 模型的输出为每百万 3.012 美元、输入 0.018;批处理降到 0.072;长上下文拿走批处理;智能体循环重发历史,一百轮对话中输出只占成本的 6.1%,缓存把 0.6137 降到 0.0708。

数据口径:本书不是投资或财务建议;全部数据为程序生成的合成数据。 第七章与《史记·货殖列传》「贪贾三之,廉贾五之」与计然「无息币」并读,把利润率与周转率的读法明确标为诠释。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 与结论吻合的打印值: 0.018, 0.072, 3.012

8B: KV cache 131,072 bytes per token; prefill 0.018 dollars per
million input tokens
batch context decode $/M memory used KV share of
output tokens bytes read
12000 3.012 20.3% 1.6%
82000 0.419 22.6% 11.6%
322000 0.141 30.5% 34.4%
1282000 0.072 61.9% 67.7%
132000 3.740 25.2% 20.8%
832000 1.147 61.9% 67.7%
3232000 does not fit in memory
12832000 does not fit in memory
70B: KV cache 327,680 bytes per token; prefill 0.156 dollars per
million input tokens
batch context decode $/M memory used KV share of
output tokens bytes read
12000 26.047 44.0% 0.5%
82000 3.362 45.4% 3.6%
322000 0.932 50.3% 13.0%
1282000 0.324 70.0% 37.5%
132000 27.868 47.0% 7.0%
832000 5.183 70.0% 37.5%
3232000 does not fit in memory
12832000 does not fit in memory
Reading: output tokens cost far more than input tokens at small batches,
because every decode step re-reads the weights. Batching spreads that read
over many users until the KV cache - which grows with context - takes over
the memory and the bandwidth. Long contexts cap the batch, and so the price.
查看完整验算程序
# Chapter 4. What a token costs to serve. An illustrative accelerator: 1.0e15
# dense FLOP/s, 3.0e12 bytes/s of memory bandwidth, 80e9 bytes of memory, 2.00
# dollars an hour. Two illustrative dense transformer sizes with grouped-query
# attention, weights in 16-bit (2 bytes per parameter):
#   "8B":  8e9 parameters, 32 layers, 8 KV heads of dimension 128, on 1 device
#   "70B": 70e9 parameters, 80 layers, 8 KV heads of dimension 128, on 4
#   devices
# KV cache per token = 2 (K and V) x layers x KV heads x head dim x 2 bytes.
# PREFILL (reading the prompt) is compute-bound: about 2 x params FLOPs per
# token, at 50 % utilisation. DECODE (writing the answer) runs one step per
# output token for the whole batch; each step must read all weights plus every
# sequence's KV cache from memory, and also do 2 x params FLOPs per sequence;
# the step takes whichever is longer.
def model(name):
    if name == "8B":
        return dict(P=8e9, L=32, H=8, Dh=128, dev=1)
    return dict(P=70e9, L=80, H=8, Dh=128, dev=4)

FLOPS, BW, MEM, PRICE = 1.0e15, 3.0e12, 80e9, 2.00

def kv_bytes(m):
    return 2 * m["L"] * m["H"] * m["Dh"] * 2

def prefill_cost_per_m(m):
    sec = 2 * m["P"] * 1e6 / (FLOPS * m["dev"] * 0.5)
    return sec * PRICE * m["dev"] / 3600

def decode(m, batch, ctx):
    w = 2 * m["P"]
    kv = batch * ctx * kv_bytes(m)
    if w + kv > MEM * m["dev"]:
        return None
    t_mem = (w + kv) / (BW * m["dev"])
    t_flop = 2 * m["P"] * batch / (FLOPS * m["dev"])
    t = max(t_mem, t_flop)
    return (
        t / batch * 1e6 * PRICE * m["dev"] / 3600,
        (w + kv) / (MEM * m["dev"]),
        kv / (w + kv),
    )

if __name__ == "__main__":
    for name in ("8B", "70B"):
        m = model(name)
        print("{}: KV cache {:,} bytes per token; prefill {:.3f} dollars per"
              .format(name, kv_bytes(m), prefill_cost_per_m(m)))
        print("   million input tokens")
        print(
            "   batch   context   decode $/M      memory used   KV share of"
        )
        print("                     output tokens                 bytes read")
        for ctx in (2000, 32000):
            for batch in (1, 8, 32, 128):
                r = decode(m, batch, ctx)
                if r is None:
                    print(
                        "   {:>5}   {:>6}    does not fit in memory".format(
                            batch, ctx
                        )
                    )
                else:
                    print(
                        "   {:>5}   {:>6}    {:9.3f}          {:6.1%}       "
                        " {:6.1%}".format(batch, ctx, r[0], r[1], r[2])
                    )
        print()
    print(
        "Reading: output tokens cost far more than input tokens at small"
        " batches,"
    )
    print(
        "because every decode step re-reads the weights. Batching spreads"
        " that read"
    )
    print(
        "over many users until the KV cache - which grows with context -"
        " takes over"
    )
    print(
        "the memory and the bandwidth. Long contexts cap the batch, and so"
        " the price."
    )

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《订阅制单位经济学、净收入留存与人工智能推理标度律》第 4 章「一个 Token 的成本:预填充、解码与内存墙」(版本 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090164

DOI: 10.5281/zenodo.24090164 · 阅读本章所在专著 · 5DL-B-058 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026058,
  author    = {Fan, Yafu},
  title     = {Subscription Unit Economics, Net Revenue Retention and AI Inference Scaling Laws},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.24090164},
  url       = {https://doi.org/10.5281/zenodo.24090164}
}

本书其他章节结论