5D Library › High-Frequency Limit Order Book Microstructure, Hawkes Self-Exciting Point Processes and Adverse-Selection Flow Toxicity · 中文
Chapter 4 Measuring Toxicity: the Probability of Informed Trading, VPIN and the Flash-Crash Dispute
From model to measurement
The Glosten–Milgrom model of Chapter 1 tells us that the spread depends on the share of informed traders; but cannot be seen. This chapter discusses two ways of estimating it from trade data, the probability of informed trading (PIN) of Easley, Kiefer, O’Hara and Paperman (1996) and the volume-synchronised probability of informed trading (VPIN) of Easley, López de Prado and O’Hara (2012); then the dispute over VPIN and the flash crash of 6 May 2010; and finally a spread-decomposition model showing how “toxicity” makes liquidity providers lose money and even withdraw.
PIN: a mixture model of daily buy and sell counts
Evidence
The model of Easley and co-authors treats each trading day as a draw (Easley et al. 1996): with probability there is an information event that day, bad news with probability ; uninformed buys and sells arrive as Poisson processes with intensities and ; on good-news days informed traders buy with additional intensity , and on bad-news days they sell with intensity . The probability of informed trading is defined as
The parameters are estimated by maximum likelihood from daily counts of buys and sells ; the likelihood is a mixture of three products of Poisson probabilities. Using NYSE stocks they found that actively traded stocks have lower PIN and that information-based trading has a significant effect on spreads. Easley, Hvidkjaer and O’Hara then added PIN to the Fama–French model for NYSE stocks over 1983–1998 and estimated that a difference of 10 percentage points in PIN between two stocks corresponds to a difference in expected returns of 2.5 % a year (Easley et al. 2002). Duarte and Young split PIN into a part related to asymmetric information and a part related to illiquidity and found that only the latter is priced, concluding that the link between PIN and returns comes from liquidity effects unrelated to information (Duarte and Young 2009).
“PIN measures informed trading” is therefore itself a claim that needs qualifying: it measures how unbalanced buy and sell counts are from day to day, and the model interprets that imbalance as information events. The program below first checks the estimator’s sampling error in a simulation with known parameters, then demonstrates a published computational problem.
Calculation
# PIN model of Easley, Kiefer, O'Hara & Paperman (1996). Each day an information event
# occurs with probability alpha; it is bad news with probability delta. Uninformed buys
# and sells are Poisson(eb), Poisson(es); informed traders add Poisson(mu) on one side.
# PIN = alpha mu / (alpha mu + eb + es). Daily buy and sell counts are simulated
# (60 days), the likelihood is maximised (Nelder-Mead on transformed parameters), and
# the sampling spread of the estimate is checked. Also: why the textbook form of the
# likelihood fails for heavily traded stocks (floating-point overflow; Lin & Ke 2011).
import math, random, statistics
def poisson(rng, m):
if m > 50: # normal approximation, large means
return max(0, round(rng.gauss(m, math.sqrt(m))))
k, p, L = 0, rng.random(), math.exp(-m)
while p > L:
k += 1; p *= rng.random()
return k
def simulate(rng, a, d, mu, eb, es, days=60, flags=None):
out = []
for _ in range(days):
ev = rng.random() < a
bad = ev and rng.random() < d; good = ev and not bad
out.append((poisson(rng, eb + mu * good), poisson(rng, es + mu * bad)))
if flags is not None:
flags.append(ev)
return out
def naive_lik(th, data): # the textbook product of terms
a, d, mu, eb, es = th; L = 1.0
P = lambda k, m: math.exp(-m) * m ** k / math.factorial(k)
for B, S in data:
L *= ((1 - a) * P(B, eb) * P(S, es) + a * d * P(B, eb) * P(S, es + mu)
+ a * (1 - d) * P(B, eb + mu) * P(S, es))
return L
def loglik(th, data, lg):
a, d, mu, eb, es = th; ll = 0.0
for (B, S), c in zip(data, lg):
t = (math.log(1 - a) + B * math.log(eb) - eb + S * math.log(es) - es,
math.log(a * d) + B * math.log(eb) - eb + S * math.log(es + mu) - es - mu,
math.log(a * (1 - d)) + B * math.log(eb + mu) - eb - mu
+ S * math.log(es) - es)
m = max(t); ll += m + math.log(sum(math.exp(x - m) for x in t)) - c
return ll
def unpack(z):
s = lambda x: 1 / (1 + math.exp(-max(-30, min(30, x))))
return s(z[0]), s(z[1]), math.exp(z[2]), math.exp(z[3]), math.exp(z[4])
def nelder_mead(f, x0, it=300, step=0.5):
n = len(x0)
pts = [list(x0)] + [[x + (step if i == j else 0) for j, x in enumerate(x0)]
for i in range(n)]
vals = [f(p) for p in pts]
for _ in range(it):
o = sorted(range(n + 1), key=lambda i: vals[i]); pts = [pts[i] for i in o]
vals = [vals[i] for i in o]
c = [sum(p[i] for p in pts[:-1]) / n for i in range(n)]
r = [2 * c[i] - pts[-1][i] for i in range(n)]; fr = f(r)
if fr < vals[0]:
e = [3 * c[i] - 2 * pts[-1][i] for i in range(n)]; fe = f(e)
pts[-1], vals[-1] = (e, fe) if fe < fr else (r, fr)
elif fr < vals[-2]:
pts[-1], vals[-1] = r, fr
else:
k = [(c[i] + pts[-1][i]) / 2 for i in range(n)]; fk = f(k)
if fk < vals[-1]:
pts[-1], vals[-1] = k, fk
else:
pts = [pts[0]] + [[(pts[0][i] + p[i]) / 2 for i in range(n)]
for p in pts[1:]]
vals = [vals[0]] + [f(p) for p in pts[1:]]
return pts[vals.index(min(vals))]
def fit(data):
lg = [math.lgamma(B + 1) + math.lgamma(S + 1) for B, S in data]
mb = statistics.fmean(B for B, _ in data); ms = statistics.fmean(S for _, S in data)
best = None
for a0 in (0.2, 0.6): # two starting points
z0 = [math.log(a0 / (1 - a0)), 0.0, math.log(0.5 * (mb + ms) / 2 + 1),
math.log(0.8 * mb + 1), math.log(0.8 * ms + 1)]
z = nelder_mead(lambda z: -loglik(unpack(z), data, lg), z0)
v = loglik(unpack(z), data, lg)
if best is None or v > best[0]:
best = (v, unpack(z))
return best[1]
pin = lambda a, d, mu, eb, es: a * mu / (a * mu + eb + es)
if __name__ == "__main__":
rng = random.Random(1996)
true = (0.4, 0.5, 40.0, 30.0, 30.0)
print(f"truth: alpha 0.4, delta 0.5, mu 40, eps_b = eps_s = 30: PIN {pin(*true):.4f}")
est = [fit(simulate(rng, *true)) for _ in range(100)]
p = sorted(pin(*e) for e in est)
print(f"100 samples of 60 days: PIN median {p[50]:.4f}, 5-95 % range"
f" {p[5]:.4f} to {p[94]:.4f}")
a = [e[0] for e in est]
print(f" alpha median {statistics.median(a):.3f}; estimates with alpha < 0.05 or"
f" > 0.95: {sum(x < 0.05 or x > 0.95 for x in a)}")
busy = (0.4, 0.5, 4000.0, 3000.0, 3000.0)
fl = []; data = simulate(rng, *busy, flags=fl)
print("60 days with about 6,000 trades a day (mu 4,000, eps 3,000 each side):")
try:
print(" textbook likelihood:", naive_lik(busy, data))
except OverflowError as e:
print(f" textbook likelihood: OverflowError {e}")
e = fit(data)
print(f" log-sum-exp form: alpha {e[0]:.3f}, mu {e[2]:,.0f}, PIN {pin(*e):.4f}"
f" (truth {pin(*busy):.4f})")
k = sum(fl)
print(f" event days actually drawn in this sample: {k} of 60 = {k / 60:.3f};"
f" PIN at that alpha {pin(k / 60, *busy[1:]):.4f}")
# truth: alpha 0.4, delta 0.5, mu 40, eps_b = eps_s = 30: PIN 0.2105
# 100 samples of 60 days: PIN median 0.2084, 5-95 % range 0.1692 to 0.2523
# alpha median 0.400; estimates with alpha < 0.05 or > 0.95: 0
# 60 days with about 6,000 trades a day (mu 4,000, eps 3,000 each side):
# textbook likelihood: OverflowError (34, 'Numerical result out of range')
# log-sum-exp form: alpha 0.466, mu 4,011, PIN 0.2380 (truth 0.2105)
# event days actually drawn in this sample: 28 of 60 = 0.467; PIN at that alpha 0.2373With a true PIN of 0.2105, 100 samples of 60 trading days each give a median estimate of 0.2084 and a 5–95 % range of 0.1692 to 0.2523: even if the model is exactly right, two and a half months of data pin PIN down only to about ±0.04. With these parameters no estimate fell on the boundary (none with below 0.05 or above 0.95).
The second part switches to about 6,000 trades a day. The textbook form of the likelihood, which computes products of directly, overflows immediately in Python (OverflowError); in other languages it may silently become zero or infinity, and the optimiser then cannot find the true solution. Lin and Ke showed that such floating-point overflow or underflow can eliminate the feasible solution corresponding to the true parameters; they estimated that about 44 % of PIN estimates from recent stock data may be biased downward for this reason, more so for active stocks, and proposed a corrected formulation (Lin and Ke 2011). The program works in logarithms and combines the three components by “log-sum-exp”, obtaining a PIN of 0.2380. It is above the true 0.2105 not because of the computation but because this sample happens to contain 28 event days (a share of 0.467), and the PIN at that share is 0.2373. When there are many trades per day, event days can be identified almost perfectly, and the remaining error comes mainly from sampling how many of the 60 days had an event.
VPIN: measuring order-flow toxicity in volume time
Evidence
Easley, López de Prado and O’Hara define order flow as toxic when it adversely selects market makers, who may not realise that they are providing liquidity at a loss (Easley et al. 2012). VPIN is computed in volume time: trades are poured in sequence into “volume buckets” of size (they used one-fiftieth of average daily volume); the buy volume and sell volume of each bucket are estimated; and the last buckets are averaged:
Buy and sell volumes are estimated by “bulk volume classification”: the volume of each one-minute (or other) bar is allocated according to the standardised price change, , where is the standard normal distribution function. No unobservable parameters need be estimated and no numerical optimisation is needed. They report that VPIN is positively correlated with subsequent short-term volatility, which they call “toxicity-induced volatility”. Easley and co-authors later compared bulk classification with trade-by-trade classification (such as the Lee–Ready algorithm, which combines a quote rule with a tick rule (Lee and Ready 1991)) for extracting information (Easley et al. 2016).
O’Hara has argued that in high-frequency markets traders act in volume rather than clock time, so that microstructure research needs concepts such as a “volume clock” (O’Hara 2015). This is the rationale behind VPIN’s design.
The flash-crash dispute
Evidence
On the afternoon of 6 May 2010, US equity indices and index futures fell sharply within minutes and rebounded quickly. The joint report of the US Commodity Futures Trading Commission and Securities and Exchange Commission traced the start to a fundamental trader’s sell program of about 75,000 E-mini S&P 500 futures contracts: the algorithm was set to execute at 9 % of the previous minute’s volume, without regard to price or time, and finished in about 20 minutes; the E-mini fell about 3 % in roughly four minutes, from 2:41 to 2:44 p.m.; and the report describes liquidity crises at both the index-futures and the individual-stock level (U.S. Commodity Futures Trading Commission and U.S. Securities and Exchange Commission 2010). Using audit-trail data for that day and the three previous days, Kirilenko, Kyle, Samadi and Tuzun found that the most active intraday intermediaries (high-frequency traders) did not change their trading behaviour as prices fell (Kirilenko et al. 2017).
Evidence
Easley, López de Prado and O’Hara argued in 2011 that the flash crash was a product of the current market structure: order-flow toxicity rose over the hours and days before the crash, VPIN rose with it, and liquidity providers were finally forced out (Easley et al. 2011). Recomputing it, Andersen and Bondarenko concluded that VPIN is a poor predictor of short-run volatility, that it peaked after the flash crash rather than before it, and that its predictive power comes mainly from a mechanical relation with trading intensity; they reached similar conclusions about the 2012 version of VPIN (Andersen and Bondarenko 2014b). The original authors replied that the critique addressed a method they do not advocate and an analysis they never performed (Easley et al. 2014). Andersen and Bondarenko responded that VPIN is correlated with volume and return volatility by construction, that it has no incremental power to predict future volatility once current volume and volatility are controlled for, that it did not reach unusual extremes before the crash, and that the results depend heavily on the trade-classification method (Andersen and Bondarenko 2014a).
The author’s framework makes “the adverse-selection withdrawal of liquidity providers on the eve of the flash crash” part of VPIN. After checking, three things should be kept apart: the definition and computation of VPIN are clear; whether VPIN gave an unusual signal before the crash is a disputed empirical question; and “liquidity providers withdrew because of adverse selection, causing the crash” is a mechanistic interpretation. The official report and the data analysis of Kirilenko and co-authors describe a large sell order, transmission between markets and the behaviour of high-frequency intermediaries; they do not establish that causal chain. The book treats it as a hypothesis.
The program below tests a mechanical property of VPIN on simulated one-minute data (40 trading days of 390 bars). On day 20 it plants 90 minutes of informed buying (62 % of volume is buying, and the price rises as a result); on day 30, 90 minutes of a volatility shock (noise standard deviation four times larger, but buying and selling balanced and no informed trading at all). Bulk classification standardises by the standard deviation of one-minute price changes over the previous five days. For comparison the program also computes VPIN from the simulation’s “true” buy and sell volumes, which is impossible with real data.
Calculation
# VPIN (Easley, Lopez de Prado & O'Hara 2012) on SIMULATED one-minute bars: 40 days of
# 390 bars. Bulk volume classification: V_B = V Phi(dP / s), s = trailing five-day
# standard deviation of one-minute price changes; buckets of 1/50 of average daily
# volume; VPIN = mean of |V_S - V_B| / V over the last 50 buckets. Two 90-minute
# episodes are planted:
# day 20 informed flow: 62 % of volume is buying (true imbalance), price drifts up
# day 30 volatility burst: noise sd x 4, buying and selling balanced (no informed flow)
# "true VPIN" uses the simulated buy and sell volumes, which no real data set contains.
# A last block repeats the episode checks for 20 further seeds (rank among ~40 days).
import math, random, statistics
ND = statistics.NormalDist()
BARS, DAYS = 390, 40
def simulate(rng):
bars = []
for d in range(DAYS):
for m in range(BARS):
inf = d == 20 and 150 <= m < 240; burst = d == 30 and 150 <= m < 240
v = 1000 * math.exp(rng.gauss(0, 0.3))
b = 0.62 if inf else min(0.9, max(0.1, rng.gauss(0.5, 0.05)))
B = v * b; S = v - B
dp = 0.02 * (B - S) / 100 + rng.gauss(0, 0.05 * (4 if burst else 1))
bars.append((d, m, v, B, S, dp))
return bars
def vpin(bars, true=False, nb=50):
adv = sum(b[2] for b in bars) / DAYS; V = adv / nb
hist, out, cur, vb, vs = [], [], 0.0, 0.0, 0.0
s1 = s2 = 0.0; W = 5 * BARS # rolling sums for the trailing sd
for i, (d, m, v, B, S, dp) in enumerate(bars):
n = min(i, W)
s = math.sqrt(max(s2 / n - (s1 / n) ** 2, 1e-12)) if n > 30 else 0.054
fb = B / v if true else ND.cdf(dp / s)
s1 += dp; s2 += dp * dp
if i >= W:
s1 -= bars[i - W][5]; s2 -= bars[i - W][5] ** 2
left = v
while left > 0: # fill buckets of size V
take = min(left, V - cur); cur += take; left -= take
vb += take * fb; vs += take * (1 - fb)
if cur >= V - 1e-9:
hist.append(abs(vs - vb) / V); cur = vb = vs = 0.0
if len(hist) >= nb:
out.append((d, m, statistics.fmean(hist[-nb:])))
return out
def summary(series, name):
vals = sorted(x[2] for x in series)
cdf = lambda x: sum(v <= x for v in vals) / len(vals)
q90 = vals[int(0.9 * len(vals))]
days = {}
for x in series:
days[x[0]] = max(days.get(x[0], 0), x[2])
ordin = [v for k, v in days.items() if k not in (20, 21, 30, 31)]
print(f"{name}: median {vals[len(vals) // 2]:.3f}, 90th pct {q90:.3f}")
print(f" ordinary days whose maximum exceeds the 90th pct:"
f" {sum(v > q90 for v in ordin)} of {len(ordin)}")
mx = sorted(days.values(), reverse=True)
for day, lab in ((20, "informed"), (30, "vol burst")):
win = [x for x in series if x[0] == day or (x[0] == day + 1 and x[1] < 60)]
top = max(win, key=lambda x: x[2])
first = next((x for x in win if x[2] > q90), None)
when = (f"d{first[0]} m{first[1]}" if first else "never")
r = 1 + sum(v > top[2] for k, v in days.items() if k not in (day, day + 1))
print(f" {lab:9s} max {top[2]:.3f} (CDF {cdf(top[2]):.3f}, rank {r}/{len(mx)})"
f" at d{top[0]} m{top[1]}; > 90th pct from {when}")
if __name__ == "__main__":
rng = random.Random(2010); bars = simulate(rng)
bvc, tru = vpin(bars), vpin(bars, true=True)
summary(bvc, "VPIN, bulk classification"); summary(tru, "true-imbalance VPIN")
d20 = [b for b in bars if b[0] == 20 and 150 <= b[1] < 240]
d30 = [b for b in bars if b[0] == 30 and 150 <= b[1] < 240]
print(f"episode price moves: informed {sum(b[5] for b in d20):+.2f},"
f" burst {sum(b[5] for b in d30):+.2f} (daily sd of price about"
f" {0.054 * math.sqrt(BARS):.2f})")
for name, ep in (("informed", d20), ("burst", d30)):
fb = statistics.fmean(ND.cdf(b[5] / 0.054) for b in ep)
tb = sum(b[3] for b in ep) / sum(b[2] for b in ep)
ab = statistics.fmean(abs(2 * ND.cdf(b[5] / 0.054) - 1) for b in ep)
print(f" {name:8s}: true buy share {tb:.3f}; bulk-classified {fb:.3f};"
f" mean |2 Phi - 1| per bar {ab:.3f}")
print("robustness, 20 further seeds (1-20): episode window (its day and the next"
" hour)")
print(" series episode above 90th pct median rank ranked first")
rob = {}
for sd in range(1, 21):
b2 = simulate(random.Random(sd))
for tr in (False, True):
s = vpin(b2, true=tr); q = sorted(x[2] for x in s)[int(0.9 * len(s))]
dmax = {}
for x in s:
dmax[x[0]] = max(dmax.get(x[0], 0), x[2])
for day in (20, 30):
top = max(x[2] for x in s if x[0] == day or (x[0] == day + 1
and x[1] < 60))
r = 1 + sum(v > top for k, v in dmax.items() if k not in (day, day + 1))
rob.setdefault((tr, day), []).append((top > q, r))
for (tr, day), v in sorted(rob.items()):
lab = ("true" if tr else "bulk", "informed" if day == 20 else "burst")
med = statistics.median(r for _, r in v)
print(f" {lab[0]:4s} {lab[1]:8s} {sum(a for a, _ in v):2d} of 20"
f" {med:4.1f} {sum(r == 1 for _, r in v):2d} of 20")
# VPIN, bulk classification: median 0.165, 90th pct 0.193
# ordinary days whose maximum exceeds the 90th pct: 9 of 36
# informed max 0.265 (CDF 0.999, rank 1/40) at d21 m11; > 90th pct from d20 m187
# vol burst max 0.209 (CDF 0.966, rank 4/40) at d30 m388; > 90th pct from d30 m222
# true-imbalance VPIN: median 0.029, 90th pct 0.033
# ordinary days whose maximum exceeds the 90th pct: 12 of 36
# informed max 0.074 (CDF 0.993, rank 1/40) at d21 m43; > 90th pct from d20 m171
# vol burst max 0.034 (CDF 0.934, rank 8/40) at d30 m294; > 90th pct from d30 m237
# episode price moves: informed +5.02, burst +0.90 (daily sd of price about 1.07)
# informed: true buy share 0.620; bulk-classified 0.777; mean |2 Phi - 1| per bar 0.618
# burst : true buy share 0.505; bulk-classified 0.584; mean |2 Phi - 1| per bar 0.806
# robustness, 20 further seeds (1-20): episode window (its day and the next hour)
# series episode above 90th pct median rank ranked first
# bulk informed 20 of 20 1.0 20 of 20
# bulk burst 14 of 20 11.5 0 of 20
# true informed 20 of 20 1.0 20 of 20
# true burst 4 of 20 20.5 0 of 20The bulk-classified VPIN responds to the informed buying: its maximum, 0.265, is the highest of the 40 days, with an empirical distribution value (CDF) of 0.999; it crosses its 90th percentile at minute 187 of day 20 (37 minutes after the episode begins), and its peak comes after the next day’s open (day 21, minute 11), because 50 buckets are about one day’s volume and VPIN is a lagged moving average. It also rises, though by less, for the volatility shock with no informed trading at all: in this sample to 0.209 (CDF 0.966, fourth of the 40 days). The reason is in the two lines that follow the episode summaries: during the volatility shock the mean of per one-minute bar is 0.806, higher even than the 0.618 of the informed episode; once price changes become large relative to their historical standard deviation, bulk classification assigns most of a bar’s volume to one side. But the side changes at random from bar to bar, so much of this cancels within a bucket, and VPIN rises less than in the informed episode. VPIN computed from true volumes barely reacts to the volatility shock (maximum 0.034, about its 90th percentile of 0.033, eighth of the 40 days). The last block repeats the check with 20 further seeds. Both versions rank the informed episode first in all 20 runs. For the volatility shock, bulk-classified VPIN exceeds its 90th percentile in 14 of 20 runs (median rank 11.5 among about 40 days), against 4 of 20 (median rank 20.5) for true-imbalance VPIN, and neither ever ranks it first. The mechanical effect is therefore systematic but moderate: this sample’s fourth place is on the high side, and in any single sample a volatility shock may or may not stand out. Another detail: “above the 90th percentile” is not rare in itself; the maximum exceeds it on 9 of the 36 ordinary days. Figure 4 shows both versions of VPIN over the whole sample.
This program cannot settle the flash-crash dispute: its data are simulated and its parameters illustrative. It shows only that the mechanical relation Andersen and Bondarenko pointed to can indeed occur: classifying trades by price changes can make “large price swings” look like “one-sided order flow”.
Spread decomposition and the withdrawal of liquidity
A more direct way of measuring toxicity is to see where the price goes after each trade. For a trade in direction (+1 for a buy) at price when the mid is , the effective spread splits into the realized spread and the price impact , where is a horizon chosen in advance. The realized spread is what the liquidity provider actually keeps after ; the price impact measures the information in the trade. Huang and Stoll gave a general method for decomposing the spread into order-processing, inventory and adverse-selection components (Huang and Stoll 1997).
The program below is the author’s model: each period the value moves by (known in advance only to the informed) plus public noise; the dealer quotes either side of the value; in the second part noise traders are sensitive to the spread and trade with probability , and we ask whether the dealer can still find a quote that breaks even.
Calculation
# Effective spread = realized spread + price impact, in a model with toxic flow (author's
# toy model, in the spirit of Glosten-Milgrom). Each period the value moves by J = +-1
# (known in advance only to informed traders) plus public noise e ~ N(0, 1). A dealer
# quotes value +- s. A trade is informed with probability a (it buys iff J = +1);
# otherwise it is a noise trade (buy or sell with probability 1/2). Measures per trade,
# with d = +1 for a buy: effective 2 d (p - m), realized 2 d (p - m_next),
# price impact 2 d (m_next - m), where m_next is the value one period later.
# Part 2: noise traders are price-elastic, trading with probability 1 - s / 0.8, so
# the dealer must find a break-even half-spread; beyond some toxicity none exists.
import math, random, statistics
def measure(rng, a, s, n=200_000):
eff, rs, pi = [], [], []
for _ in range(n):
J = 1 if rng.random() < 0.5 else -1
d = J if rng.random() < a else (1 if rng.random() < 0.5 else -1)
m_next = J + rng.gauss(0, 1) # value now normalised to 0
p = d * s
eff.append(2 * d * p); rs.append(2 * d * (p - m_next)); pi.append(2 * d * m_next)
return statistics.fmean(eff), statistics.fmean(rs), statistics.fmean(pi), \
statistics.stdev(rs)
def dealer_profit(s, a_rate, b_rate=1.0, smax=1.5):
q = max(0.0, 1 - s / smax) # noise participation
return b_rate * q * s + a_rate * (s - 1), b_rate * q + a_rate
if __name__ == "__main__":
rng = random.Random(1997)
print("half-spread s = 0.25 (effective spread 0.50), 200,000 simulated trades each")
print(" informed share effective realized impact theory: realized impact")
for a in (0.0, 0.1, 0.25, 0.4):
e, r, p, sd = measure(rng, a, 0.25)
print(f" {a:4.2f} {e:6.3f} {r:+7.3f} {p:+6.3f}"
f" {0.5 - 2 * a:+6.3f} {2 * a:+6.3f}")
print(f" sd of one trade's realized spread (share 0.40) {sd:.2f}; with 500 trades a"
f" day,\n the daily mean has s.e. {sd / math.sqrt(500):.3f}")
print("break-even half-spread; noise traders trade with probability 1 - s / 0.8, so")
print("they leave before the spread reaches 1 (where informed trading would stop):")
print(" a s* noise participation trades per period")
for a in (0.05, 0.1, 0.2, 0.3, 0.35, 0.38, 0.4):
root = next((i / 20000 * 0.8 for i in range(1, 20001)
if dealer_profit(i / 20000 * 0.8, a, smax=0.8)[0] >= 0), None)
if root is None:
print(f" {a:4.2f} none: no quote breaks even, the market shuts down")
else:
q = 1 - root / 0.8
print(f" {a:4.2f} {root:.3f} {q:.3f} {q + a:.3f}")
# max_s [(1 - s/0.8) s + a (s - 1)] >= 0 <=> 0.2 a^2 - 0.6 a + 0.2 >= 0
a_star = (0.6 - 0.2 ** 0.5) / 0.4
print(f"threshold (closed form): a* = (0.6 - sqrt(0.2)) / 0.4 = {a_star:.4f}")
# half-spread s = 0.25 (effective spread 0.50), 200,000 simulated trades each
# informed share effective realized impact theory: realized impact
# 0.00 0.500 +0.500 -0.000 +0.500 +0.000
# 0.10 0.500 +0.294 +0.206 +0.300 +0.200
# 0.25 0.500 +0.012 +0.488 +0.000 +0.500
# 0.40 0.500 -0.295 +0.795 -0.300 +0.800
# sd of one trade's realized spread (share 0.40) 2.70; with 500 trades a day,
# the daily mean has s.e. 0.121
# break-even half-spread; noise traders trade with probability 1 - s / 0.8, so
# they leave before the spread reaches 1 (where informed trading would stop):
# a s* noise participation trades per period
# 0.05 0.051 0.937 0.987
# 0.10 0.103 0.871 0.971
# 0.20 0.215 0.732 0.932
# 0.30 0.346 0.568 0.868
# 0.35 0.432 0.460 0.810
# 0.38 0.525 0.343 0.723
# 0.40 none: no quote breaks even, the market shuts down
# threshold (closed form): a* = (0.6 - sqrt(0.2)) / 0.4 = 0.3820With the half-spread fixed at 0.25, the effective spread is always 0.50; as the informed share rises from 0 to 0.4, the price impact rises from 0 to 0.795 (theory ) and the realized spread falls from +0.500 to −0.295 (theory ), reaching zero at an informed share of 0.25. Note also the noise of measurement: the standard deviation of a single trade’s realized spread is 2.70, so with 500 trades a day the daily mean has a standard error of 0.121; telling the sign of the realized spread reliably needs far more than a day’s data.
The second part reproduces the market shutdown pointed out by Glosten and Milgrom (Glosten and Milgrom 1985). With an informed arrival rate of 0.05 the break-even half-spread is 0.051; at 0.35 it rises to 0.432 and noise-trader participation falls to 0.460; at 0.38 the break-even half-spread is 0.525. The closed form gives the threshold : above that informed arrival rate no quote can break even, the dealer stops quoting and the market shuts down. In this model the “withdrawal of liquidity” needs no panic or default by the dealer; it is a mathematical consequence of the break-even condition. But this is only a model, and it does not show that any real flash crash happened this way.
Hypothesis · not established · A toxicity threshold for withdrawal
The author’s hypothesis (not established): in real limit order books, depth at the best quotes falls systematically within a few minutes after the realized spread (60-second horizon, computed over a rolling window) turns significantly negative, and falls by more than in periods with a comparable rise in volatility but no turn in the realized spread. What would show it wrong: with stocks, periods and matching rules fixed in advance, a confidence interval for the difference in depth changes between the two kinds of period that includes zero.
Chapter summary
PIN estimates the probability of informed trading with a mixture model of daily buy and sell counts; even when the model is right, estimates from 60 days carry a sampling error of about ±0.04, and the textbook form of the likelihood overflows for active stocks, which, Lin and Ke estimated, may have biased about 44 % of recent estimates downward. Whether PIN is priced as a measure of information asymmetry is contested. VPIN computes order imbalance in volume time and is simple to compute, but bulk classification can misread a rise in volatility as one-sided order flow, consistent with the critique of Andersen and Bondarenko. “Adverse-selection withdrawal” before the flash crash is a disputed mechanistic interpretation, not an established cause. Spread decomposition shows that the realized spread turns negative as toxicity rises and that, with price-elastic noise trading, the market shuts down beyond a threshold.
References
Fan, Y. (2026). High-Frequency Limit Order Book Microstructure, Hawkes Self-Exciting Point Processes and Adverse-Selection Flow Toxicity: From the Avellaneda–Stoikov Market-Making Equations to VPIN, Read Beside Classical Chinese Thought on Market Equilibrium [高频限价订单簿微观结构、霍克斯自激点过程与逆向选择毒流] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23828065 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).