What does Fan-Out Tail Latency conclude, in numbers, about “Queueing Tails: Utilisation Stretches the Tail, and Hedges Are Load Too”?

Answer: Consider utilisation first. At a utilisation of 0.30, a call's mean response time is 7.1 milliseconds and its 99th percentile 32.9 milliseconds; at 0.70 they rise to 16.7 and 76.8 milliseconds; at 0.90 to 50.0 and 230.3 milliseconds; at 0.95 the mean is 100.0 milliseconds and the 99th percentile 460.5 milliseconds. Going from 0.5 to 0.9 multiplies every percentile by 5.0, because every percentile is inversely proportional to one minus rho. The last few points of utilisation are the most expensive: going from 0.90 to 0.95 adds only five points of load but doubles the tail.

Scope of the numbers: Service times, slow shares and costs are illustrative; all calculations are generated by the book's programs; the book is general business analysis, not individualised business advice. Marked boxes separate published evidence, runnable Python calculations (22 programs), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.30, 0.5, 0.70, 0.9, 0.90, 0.95

M/M/1, service mean 5 ms, page = 100 parallel calls
rhomean Tcall p99page p50page p99ratio
0.307.132.935.565.82.00
0.5010.046.149.892.12.00
0.7016.776.882.9153.42.00
0.8025.0115.1124.4230.12.00
0.9050.0230.3248.8460.32.00
0.95100.0460.5497.5920.52.00
page p99 / call p99 does not depend on rho: 1.999
going from rho 0.5 to 0.9 multiplies every quantile by 5.0
Show the full program
# Chapter 5. p51: M/M/1 response-time tail and the fan-out page.
# Model: Poisson arrivals at rate lam, exponential service with mean
# 5 ms (mu = 0.2 per ms, illustrative). In an M/M/1 queue under FIFO the
# response time is exponential with rate mu - lam, so
#   P(T > t) = exp(-(mu - lam) t),  call p99 = ln(100) / (mu - lam),
# and a page of N = 100 independent calls has
#   page p99 = -ln(1 - 0.99^(1/100)) / (mu - lam).
import math

MU = 0.2


def call_q(rho, q):
    return -math.log(1.0 - q) / (MU * (1.0 - rho))


def page_q(rho, q, n=100):
    return -math.log(1.0 - q ** (1.0 / n)) / (MU * (1.0 - rho))


if __name__ == "__main__":
    print("M/M/1, service mean 5 ms, page = 100 parallel calls")
    print("   rho   mean T   call p99   page p50   page p99   ratio")
    for rho in (0.3, 0.5, 0.7, 0.8, 0.9, 0.95):
        mean = 1.0 / (MU * (1.0 - rho))
        c99, g50, g99 = call_q(rho, 0.99), page_q(rho, 0.5), page_q(rho, 0.99)
        fmt = "  {:4.2f}  {:7.1f}  {:9.1f}  {:9.1f}  {:9.1f}   {:5.2f}"
        print(fmt.format(rho, mean, c99, g50, g99, g99 / c99))
    print("page p99 / call p99 does not depend on rho: {:.3f}".format(
        page_q(0.5, 0.99) / call_q(0.5, 0.99)))
    print("going from rho 0.5 to 0.9 multiplies every quantile by {:.1f}"
          .format((1.0 - 0.5) / (1.0 - 0.9)))

Single source and citation

Fan, Y. (2026). The Multiplicative Law of Fan-Out Tail Latency and Hedged Requests: How Percentile Latency Amplifies When One Page Makes Many Calls (Version 1.0.0), Chapter 5: Queueing Tails: Utilisation Stretches the Tail, and Hedges Are Load Too. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-326 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026326,
  author    = {Fan, Yafu},
  title     = {The Multiplicative Law of Fan-Out Tail Latency and Hedged Requests: How Percentile Latency Amplifies When One Page Makes Many Calls},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book