What does Order Books, Hawkes Processes and Toxic Flow conclude, in numbers, about “Measuring Toxicity: the Probability of Informed Trading, VPIN and the Flash-Crash Dispute”?
Answer: PIN estimates the probability of informed trading with a mixture model of daily buy and sell counts; even when the model is right, estimates from 60 days carry a sampling error of about ±0.04, and the textbook form of the likelihood overflows for active stocks, which, Lin and Ke estimated, may have biased about 44 % of recent estimates downward. Whether PIN is priced as a measure of information asymmetry is contested. VPIN computes order imbalance in volume time and is simple to compute, but bulk classification can misread a rise in volatility as one-sided order flow, consistent with the critique of Andersen and Bondarenko. "Adverse-selection withdrawal" before the flash crash is a disputed mechanistic interpretation, not an established cause. Spread decomposition shows that the realized spread turns negative as toxicity rises and that, with price-elastic noise trading, the market shuts down beyond a threshold.
Scope of the numbers: Why does a market maker quote a spread? Why do orders arrive in clusters? Can "toxic" order flow be measured from trade data? This book presents mathematical models for teaching and research and is not investment or trading advice. Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's models and hypotheses; all data in the book are fictional or simulated.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 60
truth: alpha 0.4, delta 0.5, mu 40, eps_b = eps_s = 30: PIN 0.2105 100 samples of 60 days: PIN median 0.2084, 5-95 % range 0.1692 to 0.2523 alpha median 0.400; estimates with alpha < 0.05 or > 0.95: 0 60 days with about 6,000 trades a day (mu 4,000, eps 3,000 each side): textbook likelihood: OverflowError (34, 'Numerical result out of range') log-sum-exp form: alpha 0.466, mu 4,011, PIN 0.2380 (truth 0.2105) event days actually drawn in this sample: 28 of 60 = 0.467; PIN at that alpha 0.2373
Key formula
Show the full program
# PIN model of Easley, Kiefer, O'Hara & Paperman (1996). Each day an information event
# occurs with probability alpha; it is bad news with probability delta. Uninformed buys
# and sells are Poisson(eb), Poisson(es); informed traders add Poisson(mu) on one side.
# PIN = alpha mu / (alpha mu + eb + es). Daily buy and sell counts are simulated
# (60 days), the likelihood is maximised (Nelder-Mead on transformed parameters), and
# the sampling spread of the estimate is checked. Also: why the textbook form of the
# likelihood fails for heavily traded stocks (floating-point overflow; Lin & Ke 2011).
import math, random, statistics
def poisson(rng, m):
if m > 50: # normal approximation, large means
return max(0, round(rng.gauss(m, math.sqrt(m))))
k, p, L = 0, rng.random(), math.exp(-m)
while p > L:
k += 1; p *= rng.random()
return k
def simulate(rng, a, d, mu, eb, es, days=60, flags=None):
out = []
for _ in range(days):
ev = rng.random() < a
bad = ev and rng.random() < d; good = ev and not bad
out.append((poisson(rng, eb + mu * good), poisson(rng, es + mu * bad)))
if flags is not None:
flags.append(ev)
return out
def naive_lik(th, data): # the textbook product of terms
a, d, mu, eb, es = th; L = 1.0
P = lambda k, m: math.exp(-m) * m ** k / math.factorial(k)
for B, S in data:
L *= ((1 - a) * P(B, eb) * P(S, es) + a * d * P(B, eb) * P(S, es + mu)
+ a * (1 - d) * P(B, eb + mu) * P(S, es))
return L
def loglik(th, data, lg):
a, d, mu, eb, es = th; ll = 0.0
for (B, S), c in zip(data, lg):
t = (math.log(1 - a) + B * math.log(eb) - eb + S * math.log(es) - es,
math.log(a * d) + B * math.log(eb) - eb + S * math.log(es + mu) - es - mu,
math.log(a * (1 - d)) + B * math.log(eb + mu) - eb - mu
+ S * math.log(es) - es)
m = max(t); ll += m + math.log(sum(math.exp(x - m) for x in t)) - c
return ll
def unpack(z):
s = lambda x: 1 / (1 + math.exp(-max(-30, min(30, x))))
return s(z[0]), s(z[1]), math.exp(z[2]), math.exp(z[3]), math.exp(z[4])
def nelder_mead(f, x0, it=300, step=0.5):
n = len(x0)
pts = [list(x0)] + [[x + (step if i == j else 0) for j, x in enumerate(x0)]
for i in range(n)]
vals = [f(p) for p in pts]
for _ in range(it):
o = sorted(range(n + 1), key=lambda i: vals[i]); pts = [pts[i] for i in o]
vals = [vals[i] for i in o]
c = [sum(p[i] for p in pts[:-1]) / n for i in range(n)]
r = [2 * c[i] - pts[-1][i] for i in range(n)]; fr = f(r)
if fr < vals[0]:
e = [3 * c[i] - 2 * pts[-1][i] for i in range(n)]; fe = f(e)
pts[-1], vals[-1] = (e, fe) if fe < fr else (r, fr)
elif fr < vals[-2]:
pts[-1], vals[-1] = r, fr
else:
k = [(c[i] + pts[-1][i]) / 2 for i in range(n)]; fk = f(k)
if fk < vals[-1]:
pts[-1], vals[-1] = k, fk
else:
pts = [pts[0]] + [[(pts[0][i] + p[i]) / 2 for i in range(n)]
for p in pts[1:]]
vals = [vals[0]] + [f(p) for p in pts[1:]]
return pts[vals.index(min(vals))]
def fit(data):
lg = [math.lgamma(B + 1) + math.lgamma(S + 1) for B, S in data]
mb = statistics.fmean(B for B, _ in data); ms = statistics.fmean(S for _, S in data)
best = None
for a0 in (0.2, 0.6): # two starting points
z0 = [math.log(a0 / (1 - a0)), 0.0, math.log(0.5 * (mb + ms) / 2 + 1),
math.log(0.8 * mb + 1), math.log(0.8 * ms + 1)]
z = nelder_mead(lambda z: -loglik(unpack(z), data, lg), z0)
v = loglik(unpack(z), data, lg)
if best is None or v > best[0]:
best = (v, unpack(z))
return best[1]
pin = lambda a, d, mu, eb, es: a * mu / (a * mu + eb + es)
if __name__ == "__main__":
rng = random.Random(1996)
true = (0.4, 0.5, 40.0, 30.0, 30.0)
print(f"truth: alpha 0.4, delta 0.5, mu 40, eps_b = eps_s = 30: PIN {pin(*true):.4f}")
est = [fit(simulate(rng, *true)) for _ in range(100)]
p = sorted(pin(*e) for e in est)
print(f"100 samples of 60 days: PIN median {p[50]:.4f}, 5-95 % range"
f" {p[5]:.4f} to {p[94]:.4f}")
a = [e[0] for e in est]
print(f" alpha median {statistics.median(a):.3f}; estimates with alpha < 0.05 or"
f" > 0.95: {sum(x < 0.05 or x > 0.95 for x in a)}")
busy = (0.4, 0.5, 4000.0, 3000.0, 3000.0)
fl = []; data = simulate(rng, *busy, flags=fl)
print("60 days with about 6,000 trades a day (mu 4,000, eps 3,000 each side):")
try:
print(" textbook likelihood:", naive_lik(busy, data))
except OverflowError as e:
print(f" textbook likelihood: OverflowError {e}")
e = fit(data)
print(f" log-sum-exp form: alpha {e[0]:.3f}, mu {e[2]:,.0f}, PIN {pin(*e):.4f}"
f" (truth {pin(*busy):.4f})")
k = sum(fl)
print(f" event days actually drawn in this sample: {k} of 60 = {k / 60:.3f};"
f" PIN at that alpha {pin(k / 60, *busy[1:]):.4f}")Single source and citation
Fan, Y. (2026). High-Frequency Limit Order Book Microstructure, Hawkes Self-Exciting Point Processes and Adverse-Selection Flow Toxicity (Version 1.0.0), Chapter 4: Measuring Toxicity: the Probability of Informed Trading, VPIN and the Flash-Crash Dispute. 5D Library. https://doi.org/10.5281/zenodo.23828065
DOI: 10.5281/zenodo.23828065 · Read the monograph · 5DL-B-045 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026045,
author = {Fan, Yafu},
title = {High-Frequency Limit Order Book Microstructure, Hawkes Self-Exciting Point Processes and Adverse-Selection Flow Toxicity},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.23828065},
url = {https://doi.org/10.5281/zenodo.23828065}
}Other chapter answers in this book
- Chapter 1 · Information Asymmetry and the Bid–Ask Spread: Glosten–Milgrom and Kyle
- Chapter 2 · Inventory Risk and Optimal Market Making: the Avellaneda–Stoikov Equations
- Chapter 3 · Why Orders Arrive in Clusters: Hawkes Processes, the Signature Plot and the Epps Effect
- Chapter 5 · Book Imbalance and Queues: Order Flow Imbalance, Queue Imbalance and Queue Position
- Chapter 6 · How to Split a Large Order: Almgren–Chriss Optimal Execution
- Chapter 7 · Reading the Classics: the Market Director, the Conspicuous Mound and the Market Exchange
- Chapter 8 · Practice and a Pre-Registered Test