关于「端到端延迟预算:分位数而非口号」,《情境感知副官的阈值与预算》给出的定量结论是什么?

结论:解析结果一目了然:至少碰上一个慢分片的概率是 \(1 - 0.99^k\)。\(k\) 为 1 时只有 0.010,\(k\) 为 10 时 0.096,\(k\) 为 50 时 0.395,\(k\) 为 100 时 0.634。对应的 99 分位从 53 毫秒涨到 564、718、942,再到 1,177 与 1,303 毫秒。扇出到一百处时,连中位数都变成了 315 毫秒,因为超过一半的请求都碰上了慢分片。

数据口径:语音端点等待是延迟的地板,示意架构下端到端延迟中位数 1,083 毫秒、95 分位 1,540 毫秒,一秒内完成的只有 0.323,「零延迟」「人类开口前」「电影级」都被改写为分位数与推翻条件。 书中参数为示意,数据为合成;全部计算由书中 23 段标准库程序生成。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 1073, 1083, 1540, 160, 1820, 219

A. baseline stages, msp50p95p99
frame + features202020
endpoint wait260328404
speech recognition160284360
retrieval70219353
answer first token3797971,073
speech synthesis110179218
delivery355363
total1,0831,5401,820
B. variants, total ms p50 p95 p99 <=1s <=1.5s
baseline 1,083 1,540 1,820 0.323 0.938
streaming 966 1,411 1,694 0.569 0.970
stream+card 846 1,283 1,562 0.758 0.986
stream+card+h120 726 1,163 1,442 0.873 0.993
查看完整验算程序
# Chapter 3. p31: end-to-end latency budget by Monte Carlo.
# Stage time = fixed + median * exp(sigma * Z), Z standard normal, so
# each stage is a shifted log-normal. Stages run in sequence from the
# moment the user stops speaking to the first cue the user perceives.
# The endpoint wait carries the 240 ms hangover chosen in Chapter 2.
# All medians and sigmas (ms) are illustrative, not measurements of
# any product. 20,000 draws, fixed seed.
import math
import random

N = 20000
BASE = [  # name, fixed ms, median ms, sigma
    ("frame + features", 20.0, 0.0, 0.0),
    ("endpoint wait", 240.0, 20.0, 0.9),
    ("speech recognition", 0.0, 160.0, 0.35),
    ("retrieval", 0.0, 70.0, 0.7),
    ("answer first token", 0.0, 380.0, 0.45),
    ("speech synthesis", 0.0, 110.0, 0.3),
    ("delivery", 0.0, 35.0, 0.25),
]


def variant(name):
    st = [list(s) for s in BASE]
    if name in ("streaming", "stream+card", "stream+card+h120"):
        st[2] = ["speech recognition", 0.0, 50.0, 0.4]
    if name in ("stream+card", "stream+card+h120"):
        st[5] = ["speech synthesis", 0.0, 0.0, 0.0]
        st[6] = ["delivery", 0.0, 30.0, 0.25]
    if name == "stream+card+h120":
        st[1] = ["endpoint wait", 120.0, 20.0, 0.9]
    return st


def pct(xs, q):
    s = sorted(xs)
    return s[int(round(q * (len(s) - 1)))]


def simulate(stages, rng):
    totals, per = [], [[] for _ in stages]
    for _ in range(N):
        parts = []
        for j, (_, fx, med, sg) in enumerate(stages):
            v = fx + (med * math.exp(sg * rng.gauss(0.0, 1.0))
                      if med > 0 else 0.0)
            parts.append(v)
            per[j].append(v)
        totals.append(math.fsum(parts))
    return totals, per


if __name__ == "__main__":
    rng = random.Random(40931)
    totals, per = simulate(variant("baseline"), rng)
    print("A. baseline stages, ms        p50     p95     p99")
    for (name, _, _, _), xs in zip(BASE, per):
        print("  {:<24} {:>7,.0f} {:>7,.0f} {:>7,.0f}".format(
            name, pct(xs, 0.5), pct(xs, 0.95), pct(xs, 0.99)))
    print("  {:<24} {:>7,.0f} {:>7,.0f} {:>7,.0f}".format(
        "total", pct(totals, 0.5), pct(totals, 0.95), pct(totals, 0.99)))
    print("B. variants, total ms      p50     p95     p99  <=1s   <=1.5s")
    for name in ("baseline", "streaming", "stream+card",
                 "stream+card+h120"):
        rng = random.Random(40931)
        t, _ = simulate(variant(name), rng)
        a = 0
        b = 0
        for v in t:
            a += v <= 1000.0
            b += v <= 1500.0
        print("  {:<20} {:>7,.0f} {:>7,.0f} {:>7,.0f}  {:.3f}  {:.3f}"
              .format(name, pct(t, 0.5), pct(t, 0.95), pct(t, 0.99),
                      a / N, b / N))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《全天候多模态情境感知智能副官、低延迟视听记忆流与主动预判辅助母典》第 3 章「端到端延迟预算:分位数而非口号」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-409 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026409,
  author    = {Fan, Yafu},
  title     = {Ambient Multimodal Cognitive Adjutants: Low-Latency Audio-Visual Memory Streams, Proactive Intent Anticipation and Real-Time Decision Support},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论