5D Library › High-Frequency Limit Order Book Microstructure, Hawkes Self-Exciting Point Processes and Adverse-Selection Flow Toxicity · 中文
Chapter 8 Practice and a Pre-Registered Test
Note
The checklist and experimental protocol in this chapter are for research and self-examination, not investment or trading advice. The checklist cannot tell you how to quote or trade; it can only help you write down your assumptions before you decide. The trading days, orders, fills and strategies in the book are fictional or simulated; any real trading can lose money.
Part 1: a four-step checklist
Chapters 1 to 6 can be condensed into a four-step checklist for examining a day of market making or execution. The limits in it (for instance, “flag when the mean realized spread plus two standard errors is still below zero”) are examples of rules the user should write down in advance, not recommended values.
- Adverse selection. Compute the realized spread and price impact over a fixed horizon (say 60 seconds) separately for each side of fills. Is the realized spread significantly negative on either side? Are the two sides markedly asymmetric (Chapters 1 and 4)?
- Order flow. Is the arrival intensity of market orders in one direction significantly higher than in a reference period? Are arrivals clustered (Fano factor)? Check at the same time whether volatility and volume are changing too, since any trade-based toxicity indicator can move with them mechanically (Chapters 3 and 4).
- Inventory. Split profit and loss into spread income and inventory profit and loss. Does the inventory loss exceed the spread income? Does maximum inventory exceed its limit (Chapter 2)?
- Execution. For the institution’s own parent orders, how does the implementation shortfall compare with the pre-trade estimate (expectation and standard deviation)? Does the difference exceed a multiple written down in advance (Chapter 6)?
The program below applies the checklist to a fictional trading day: 23,400 one-second steps; the market maker quotes 2 ticks either side of the mid, skewed by inventory; in the last 30 minutes an informed seller sends a 100-share market sell with probability 0.05 per second, each followed by a permanent fall of 1.5 ticks in the mid; and on the same day the institution sells a 30,000-share parent order in slices, one every 60 seconds.
Calculation
# The four-step check applied to a FICTIONAL market-making day (23,400 one-second steps,
# mid starts at 100.00, tick 0.01, public noise 0.6 tick per second). The maker quotes
# +-2 ticks around the mid, skewed by 0.2 tick per 100 shares of inventory, 100 shares
# per fill. From 15:30 (step 21,600) an informed seller sends extra 100-share market
# sells at rate 0.05 per second, each followed by a permanent 1.5-tick fall of the mid.
# The desk also sells a parent order of 30,000 shares in 300 slices, one per 60 s, from
# 10:00, each crossing the spread (half-spread 1 tick) and lowering the mid 0.1 tick.
# Flag thresholds are written down BEFORE looking at the day.
import math, random, statistics
TICK, STEPS, TOX = 0.01, 23_400, 21_600
def day(rng):
mid, q, fills, mids, inv, slices, sells = 100.0, 0, [], [], [], [], []
for t in range(STEPS):
mid += 0.6 * TICK * rng.gauss(0, 1); mids.append(mid)
skew = 0.2 * TICK * q / 100
bid, ask = mid - 2 * TICK - skew, mid + 2 * TICK - skew
lam_b = 0.10 * math.exp(-(mid - bid) / TICK + 2) # noise sells hit our bid
lam_a = 0.10 * math.exp(-(ask - mid) / TICK + 2) # noise buys lift our ask
if rng.random() < lam_b:
q += 100; fills.append((t, +1, bid)); sells.append(t)
if rng.random() < lam_a:
q -= 100; fills.append((t, -1, ask))
if t >= TOX and rng.random() < 0.05: # informed seller
q += 100; fills.append((t, +1, bid)); sells.append(t); mid -= 1.5 * TICK
if 1800 <= t < 1800 + 300 * 60 and (t - 1800) % 60 == 0:
slices.append((t, mid, mid - TICK)); mid -= 0.1 * TICK # desk's own slice
inv.append(q)
return fills, mids, inv, slices, sells
def markout(fills, mids, h):
rs, pi = [], []
for t, d, p in fills:
D = -d # the taker's direction (+1 = buy)
m0, m1 = mids[t], mids[min(t + h, STEPS - 1)]
rs.append(2 * D * (p - m1) / TICK); pi.append(2 * D * (m1 - m0) / TICK)
return rs, pi
if __name__ == "__main__":
rng = random.Random(45); fills, mids, inv, slices, sells = day(rng)
print(f"fictional day: {len(fills):,} fills; close {mids[-1]:.2f};"
f" final inventory {inv[-1]:+,} shares")
print("1 adverse selection: 60-s realized spread by side, ticks per fill"
"\n (flag if mean + 2 s.e. < 0)")
for lab, lo, hi in (("09:30-15:30", 0, TOX), ("15:30-16:00", TOX, STEPS)):
for side, nm in ((+1, "our bid hit"), (-1, "our ask hit")):
rs, pi = markout([f for f in fills if lo <= f[0] < hi and f[1] == side],
mids, 60)
m, se = statistics.fmean(rs), statistics.stdev(rs) / math.sqrt(len(rs))
print(f" {lab} {nm}: {len(rs):4d} fills, realized {m:+6.2f} (s.e. {se:.2f})"
f" -> {'FLAG' if m + 2 * se < 0 else 'ok'}")
dr = (mids[TOX - 1] - mids[0]) / TICK
print(f" mid drift 09:30-15:30 {dr:+.0f} ticks ({dr / 360:+.2f} per 60 s);"
f" desk's own slices {-0.1 * len(slices):+.0f}")
print("2 flow: market sells per minute, last 30 min against 09:30-15:00"
" (flag if z > 3)")
per = [0] * (STEPS // 60)
for t in sells:
per[t // 60] += 1
ref, now = per[:330], per[360:]
m, sd = statistics.fmean(ref), statistics.stdev(ref)
z = (statistics.fmean(now) - m) / (sd / math.sqrt(len(now)))
fano = statistics.variance(ref) / m
print(f" reference {m:.2f} (Fano factor {fano:.2f}), now"
f" {statistics.fmean(now):.2f}, z = {z:+.1f} -> {'FLAG' if z > 3 else 'ok'}")
v1 = statistics.pstdev(b - a for a, b in zip(mids[:TOX], mids[1:TOX])) / TICK
v2 = statistics.pstdev(b - a for a, b in zip(mids[TOX:], mids[TOX + 1:])) / TICK
print(f" read with: 1-s volatility {v1:.2f} -> {v2:.2f} ticks")
print("3 inventory: spread capture against inventory P&L"
" (flag if inventory loss > capture)")
for lab, lo, hi in (("09:30-15:30", 0, TOX), ("15:30-16:00", TOX, STEPS - 1)):
cap = sum(d * (mids[t] - p) * 100 for t, d, p in fills if lo <= t < hi)
invp = sum(inv[t] * (mids[t + 1] - mids[t]) for t in range(lo, hi))
print(f" {lab}: capture ${cap:,.0f}, inventory ${invp:+,.0f},"
f" max |q| {max(abs(x) for x in inv[lo:hi]):,}"
f" -> {'FLAG' if -invp > cap else 'ok'}")
print("4 execution: parent sell order against its pre-trade estimate"
" (flag if |z| > 2)")
arr, n = slices[0][1], 100
sf = sum(n * (arr - px) for _, _, px in slices)
spr = sum(n * (m - px) for _, m, px in slices)
pre = 300 * n * TICK + n * 0.1 * TICK * 299 * 300 / 2
sd = 0.6 * TICK * math.sqrt(sum((n * (300 - k)) ** 2 * 60 for k in range(1, 301)))
z = (sf - pre) / sd
print(f" shortfall ${sf:,.0f} (spread ${spr:,.0f}); estimate ${pre:,.0f},"
f" sd ${sd:,.0f}; z = {z:+.2f} -> {'FLAG' if abs(z) > 2 else 'ok'}")
# fictional day: 5,004 fills; close 96.58; final inventory +0 shares
# 1 adverse selection: 60-s realized spread by side, ticks per fill
# (flag if mean + 2 s.e. < 0)
# 09:30-15:30 our bid hit: 2257 fills, realized +2.67 (s.e. 0.19) -> ok
# 09:30-15:30 our ask hit: 2256 fills, realized +4.99 (s.e. 0.19) -> ok
# 15:30-16:00 our bid hit: 245 fills, realized -7.67 (s.e. 0.65) -> FLAG
# 15:30-16:00 our ask hit: 246 fills, realized +14.49 (s.e. 0.63) -> ok
# mid drift 09:30-15:30 -181 ticks (-0.50 per 60 s); desk's own slices -30
# 2 flow: market sells per minute, last 30 min against 09:30-15:00 (flag if z > 3)
# reference 6.27 (Fano factor 0.60), now 8.17, z = +5.4 -> FLAG
# read with: 1-s volatility 0.60 -> 0.69 ticks
# 3 inventory: spread capture against inventory P&L (flag if inventory loss > capture)
# 09:30-15:30: capture $8,617, inventory $+118, max |q| 600 -> ok
# 15:30-16:00: capture $938, inventory $-276, max |q| 600 -> ok
# 4 execution: parent sell order against its pre-trade estimate (flag if |z| > 2)
# shortfall $26,002 (spread $300); estimate $4,785, sd $13,908; z = +1.53 -> okStep 1 flags the last 30 minutes: when our bid is hit the 60-second realized spread is −7.67 ticks (standard error 0.65), while when our ask is hit it is +14.49. Looking only at both sides together would suggest that all is well, because the inventory skew made the market maker sell more as the price fell, and the gains on the ask side mask the adverse selection on the bid side; adverse selection should be examined by direction. In the first six hours both sides are positive (+2.67 and +4.99); the difference between them comes from the day’s general downward drift in the mid (181 ticks over the six hours, about 0.5 tick per 60 seconds; 30 ticks of it, about a sixth, caused by the institution’s own parent sell order). Step 2 also flags: market sells per minute in the last 30 minutes rise from 6.27 in the reference period to 8.17 (z = +5.4); at the same time one-second volatility rises from 0.60 to 0.69 ticks, so, following Chapter 4, this signal should be read together with volatility. The Fano factor in the reference period is 0.60, below 1, because the inventory skew feeds back negatively from inventory to sell arrivals. Step 3 does not flag: in the last 30 minutes spread income is $938 and the inventory loss only $276, with maximum inventory 600 shares, within the 2,000-share limit; the skew quickly brought inventory back towards zero. Step 4 does not flag either: the parent order’s shortfall is $26,002 against a pre-trade estimate of $4,785 with a standard deviation of $13,908, z = +1.53, within two standard deviations. The lesson of the example is that each of the four steps looks at one side of the day; on inventory and execution alone the day looks normal, and the adverse selection shows up clearly only in realized spreads split by direction.
Part 2: a pre-registered simulated-matching experiment
Question and design choice
In a simulated limit order book built and frozen in advance by an independent third party, with impulsive informed order flow (“toxic pulses”) mixed in, does the Avellaneda–Stoikov strategy that skews its quotes dynamically by inventory achieve a higher cross-session Sharpe ratio of terminal net value than a strategy quoting a fixed symmetric spread, by at least the smallest effect of interest?
The author’s framework proposes comparing the two strategies’ effects on “the Sharpe ratio of terminal net value and the maximum inventory drawdown” in “a double-blind simulated limit-order-book matching experiment”. The book keeps the comparison groups and makes three design choices. First, simulation rather than real trading: what is compared is a quoting rule, and simulation lets both strategies face exactly the same order flow and pulses (pairing), which gives clean causal identification and puts no one’s money at risk; the price is that the conclusions apply only to the simulator, as discussed under “What this experiment cannot show”. Second, a single primary outcome: the difference in cross-session Sharpe ratios, with maximum inventory and within-session drawdown as secondary outcomes. The Sharpe ratio reflects both return and risk and corresponds directly to the statement of the hypothesis in Chapter 2; maximum drawdown is a single statistic of a whole path, which Chapter 8 of Book 38 showed has almost no statistical power; and maximum inventory is a source of risk rather than an outcome, useful as supporting evidence on mechanism. Third, execution by an independent third party: the simulator, the timing and direction of pulses and the random seeds are generated and sealed by researchers unconnected with the author, who submits only the code of the two strategies.
One point needs stating: the Sharpe ratio is not the objective of the Avellaneda–Stoikov model (which maximises expected exponential utility; Chapter 2 noted that the two can rank strategies differently). The protocol tests the question posed by the author’s framework, “which quoting rule has better risk-adjusted performance”, not “which rule better achieves its own optimisation objective”.
Protocol
| Element | Specification |
|---|---|
| Simulator | A multi-agent limit-order-book simulator chosen and frozen by the third party (for instance an open-source discrete-event simulator of the ABIDES kind (Byrd et al. 2020)), with noise traders, other market makers and an exchange matching by price–time priority; configuration files and version numbers are published at registration |
| Toxic pulses | Generated by the third party from a registered distribution: each session contains one pulse with probability 1/2, its duration, direction and intensity random within registered ranges; during a pulse the informed agent takes liquidity only when it is profitable; the generating script is published and the seeds are sealed until the results are locked |
| The two strategies | X: Avellaneda–Stoikov inventory-skewed quotes (, the paper’s formulas, horizon equal to the session length); Y: fixed symmetric quotes about the mid, with a spread equal to X’s mean spread in a separate set of calibration sessions. Order size, inventory limits and cancellation latency are the same for both |
| Pairing and blinding | Each session’s seeds are run once with X and once with Y; the strategy code is registered by hash and randomly labelled “A / B” by the third party, and those running and analysing the sessions do not know which is which until the results are locked (double-blind); the author has no access to the seeds or the pulse schedule |
| Primary outcome | , where SR is the cross-session mean of terminal net value (profit and loss marked at the mid at the end of the session) divided by its standard deviation; 95 % confidence interval from a bootstrap that resamples paired sessions (10,000 resamples) |
| Pre-specified qualifier | Whether X’s mean terminal net value is more than 5 % below Y’s; if so, the verdict carries the qualifier “the higher Sharpe ratio comes at the cost of a lower mean return” |
| Secondary outcomes | Mean terminal net value and its standard deviation; maximum absolute inventory within the session; maximum mark-to-market drawdown within the session; profit and loss during pulses; fill rates; Sharpe ratios separately for sessions with and without pulses; the normal-theory (Jobson–Korkie, Memmel) standard error (Jobson and Korkie 1981; Memmel 2003) and a robust test (Ledoit and Wolf 2008) for comparison; all descriptive or exploratory |
There is a single primary outcome, the “difference in Sharpe ratios”. The Sharpe ratio estimator has known biases, and its standard-error formula needs adjustment when returns are non-normal or serially correlated (Lo 2002); the Jobson–Korkie test for comparing two Sharpe ratios, as corrected by Memmel, has asymptotic variance under independent, identically distributed bivariate normal returns; Ledoit and Wolf pointed out that the formula depends on normality and recommended a bootstrap (Ledoit and Wolf 2008). The protocol relies on the paired bootstrap, with the normal formula only as a cross-check.
Guarding against data snooping
There is a single pre-fixed set of strategy parameters and a single primary outcome, and the author does no tuning on the independent simulator; the simulator configuration, pulse-generating script, strategy code and analysis code are registered publicly before any run, with hashes of the code registered alongside the protocol (Nosek et al. 2018); the third party carries out all runs and analyses; secondary and exploratory results are clearly labelled and cannot replace the primary conclusion. Book 33 discussed the statistical principles behind these practices.
Sample size
Sample size requires assumptions about the effect and the variance. The book sets them with a pilot environment: the author’s own toy model, the Avellaneda–Stoikov simulation of Chapter 2 with toxic pulses added, not the simulator the experiment will use.
Calculation
# Sample size for the Chapter 8 pre-registered simulation experiment. Arms, on the SAME
# simulated sessions (paired seeds): X = inventory-aware Avellaneda-Stoikov quotes
# (gamma 0.1), Y = fixed symmetric quotes whose spread equals X's mean spread.
# Primary outcome: Sharpe ratio of terminal session P&L across sessions, SR_X - SR_Y.
# PILOT ENVIRONMENT (a planning assumption, not the trial's simulator): the AS setting of
# p04 plus toxic pulses. With probability 1/2 a session contains one pulse of 20 steps at
# a random time; during it the mid drifts 0.15 per step in a random direction and, each
# step, an informed order hits the quote on the drift side with probability 0.5 whenever
# that quote is closer to the mid than the drift still to come.
import math, random, statistics
S0, T, SIG, DT, A, K, G = 100.0, 1.0, 2.0, 0.005, 140.0, 1.5, 0.1
NSTEP = round(T / DT); Z = statistics.NormalDist().inv_cdf
def session(seed, kind, half=None):
rng = random.Random(seed); s, x, q, sp, qmax = S0, 0.0, 0, [], 0
pulse = rng.random() < 0.5; start = rng.randrange(0, NSTEP - 20)
sign = 1 if rng.random() < 0.5 else -1
for j in range(NSTEP):
tau = T - j * DT
if kind == "X":
r = s - q * G * SIG ** 2 * tau
h = (G * SIG ** 2 * tau + 2 / G * math.log(1 + G / K)) / 2
bid, ask = r - h, r + h
else:
bid, ask = s - half, s + half
sp.append(ask - bid)
u1, u2, u3, u4 = rng.random(), rng.random(), rng.random(), rng.random()
if u1 < A * math.exp(-K * (ask - s)) * DT:
q -= 1; x += ask
if u2 < A * math.exp(-K * (s - bid)) * DT:
q += 1; x -= bid
on = pulse and start <= j < start + 20
gain = 0.15 * (start + 20 - j) # drift still to come
if on and u4 < 0.5: # informed order
if sign > 0 and ask - s < gain:
q -= 1; x += ask
elif sign < 0 and s - bid < gain:
q += 1; x -= bid
s += SIG * math.sqrt(DT) * (1 if u3 < 0.5 else -1) + (0.15 * sign if on else 0.0)
qmax = max(qmax, abs(q))
return x + q * s, qmax, statistics.fmean(sp)
def sr(v):
return statistics.fmean(v) / statistics.stdev(v)
def avar(a, b, rho): # Jobson-Korkie / Memmel
return 2 * (1 - rho) + 0.5 * (a * a + b * b) - rho * rho * a * b
if __name__ == "__main__":
n0 = 4000
X = [session(i, "X") for i in range(n0)]
half = statistics.fmean(r[2] for r in X) / 2
Y = [session(i, "Y", half) for i in range(n0)]
px, py = [r[0] for r in X], [r[0] for r in Y]
a, b, rho = sr(px), sr(py), statistics.correlation(px, py)
mx, sx, my, sy = (statistics.fmean(px), statistics.stdev(px), statistics.fmean(py),
statistics.stdev(py))
print(f"pilot ({n0:,} paired sessions): SR_X {a:.2f} (mean {mx:.1f}, sd {sx:.1f}),"
f"\n SR_Y {b:.2f} (mean {my:.1f}, sd {sy:.1f})")
print(f" correlation of paired P&L {rho:.2f}; pilot difference {a - b:+.2f};"
f" mean max|q|: X {statistics.fmean(r[1] for r in X):.1f},"
f" Y {statistics.fmean(r[1] for r in Y):.1f}")
pul = [i for i in range(n0) if random.Random(i).random() < 0.5] # pulse sessions
npl = [i for i in range(n0) if i not in set(pul)]
for lab, ids in (("with a pulse", pul), ("without", npl)):
print(f" {lab:12s} ({len(ids):,}): SR_X {sr([px[i] for i in ids]):.2f},"
f" SR_Y {sr([py[i] for i in ids]):.2f}, mean P&L X"
f" {statistics.fmean(px[i] for i in ids):.1f}, Y"
f" {statistics.fmean(py[i] for i in ids):.1f}")
sesoi = 0.10 * b
v = avar(a, b, rho); n = (Z(0.975) + Z(0.90)) ** 2 * v / sesoi ** 2
print(f"SESOI = 10 % of SR_Y = {sesoi:.2f}; asymptotic variance {v:.1f}"
f" (normal theory)")
print(f" 90 % power, two-sided alpha 0.05: {math.ceil(n):,} paired sessions")
rng = random.Random(8); m = math.ceil(n); diffs = []
for _ in range(400): # bootstrap check of the s.e.
idx = [rng.randrange(n0) for _ in range(m)]
diffs.append(sr([px[i] for i in idx]) - sr([py[i] for i in idx]))
se_b = statistics.stdev(diffs); se_n = math.sqrt(v / m)
print(f" s.e. of SR_X - SR_Y at n = {m:,}: normal theory {se_n:.3f},"
f" bootstrap {se_b:.3f}")
n_rob = math.ceil(n * (se_b / se_n) ** 2)
plan = math.ceil(n_rob * 1.1 / 100) * 100
print(f" robust n {n_rob:,}; plus 10 % for failed runs, rounded up: plan {plan:,}")
print("sensitivity, paired sessions (robust) SESOI: 5 % 10 % 20 %"
" of SR_Y")
for lab, f in (("bootstrap s.e. as estimated", 1.0), ("variance 50 % higher", 1.5)):
row = [math.ceil((Z(0.975) + Z(0.90)) ** 2 * v * (se_b / se_n) ** 2 * f
/ (s * b) ** 2) for s in (0.05, 0.10, 0.20)]
print(f" {lab:38s}" + "".join(f"{x:9,d}" for x in row))
# pilot (4,000 paired sessions): SR_X 9.06 (mean 64.5, sd 7.1),
# SR_Y 3.09 (mean 63.6, sd 20.6)
# correlation of paired P&L 0.33; pilot difference +5.97; mean max|q|: X 4.4, Y 11.6
# with a pulse (2,041): SR_X 8.33, SR_Y 2.37, mean P&L X 64.1, Y 59.0
# without (1,959): SR_X 10.09, SR_Y 5.21, mean P&L X 65.0, Y 68.4
# SESOI = 10 % of SR_Y = 0.31; asymptotic variance 44.0 (normal theory)
# 90 % power, two-sided alpha 0.05: 4,844 paired sessions
# s.e. of SR_X - SR_Y at n = 4,844: normal theory 0.095, bootstrap 0.121
# robust n 7,779; plus 10 % for failed runs, rounded up: plan 8,600
# sensitivity, paired sessions (robust) SESOI: 5 % 10 % 20 % of SR_Y
# bootstrap s.e. as estimated 31,114 7,779 1,945
# variance 50 % higher 46,671 11,668 2,917In the pilot environment (4,000 paired sessions), X has a Sharpe ratio of 9.06 (mean terminal net value 64.5, standard deviation 7.1) and Y 3.09 (63.6 and 20.6); the correlation of paired net values is 0.33 and the difference +5.97; X’s mean maximum inventory is 4.4 and Y’s 11.6. Y’s mean net value is 68.4 in sessions without a pulse and 59.0 in sessions with one; X’s is 65.0 and 64.1. These numbers are planning assumptions, not evidence: the pilot environment was built by the author, and in it inventory skewing is naturally favoured; the true difference in the independent simulator may be much smaller, or zero.
The smallest effect of interest is 10 % of Y’s Sharpe ratio, 0.31: a smaller improvement would not justify the added complexity of inventory-skewed quoting. With two-sided and 90 % power, the normal-theory formula (asymptotic variance 44.0) needs 4,844 paired sessions; but the standard error estimated by bootstrapping the pilot sessions is larger than the normal-theory one (0.121 against 0.095), because terminal net value is clearly skewed when there are pulses. Converting with the bootstrap standard error gives 7,779 paired sessions, and allowing for 10 % failed runs and rounding up, the plan is 8,600. The sensitivity table shows that a smallest effect of interest of 5 % would need 31,114, and a variance 50 % higher than estimated would need 11,668 for the 10 % effect. Simulated sessions cost far less than real trading, so sample size itself is not the bottleneck; the bottleneck is locking the specification in advance.
Following Chapter 6 of Book 33, “90 % power” means that if the true effect equals the smallest effect of interest exactly, the probability of reaching some conclusion that “X’s Sharpe ratio is higher” (a confidence interval whose lower bound is above 0) is about 90 %; in that case the strongest conclusion, “higher by at least the smallest effect of interest”, has only about a 2.5 % chance of appearing.
Decision rule
Calculation
# Decision rule, fixed before the third party runs the locked simulation. D = SR_X - SR_Y,
# the difference in cross-session Sharpe ratios of terminal P&L (X = inventory-aware AS
# quotes, Y = fixed symmetric quotes), with a 95 % CI from a paired bootstrap over
# sessions (10,000 resamples). SESOI 0.31. Gates, checked first: more than 5 % of
# sessions failed or truncated; the independent re-run disagreeing on more than 0.1 % of
# sessions; arm labels revealed before the results were locked -> NOT INTERPRETABLE.
# Pre-specified qualifier: whether X's mean P&L is more than 5 % below Y's.
SESOI = 0.31
def verdict(d, lo, hi, mean_ratio, failed=0.01, mismatch=0.0, unblinded=False):
for bad, why in ((failed > 0.05, f"{failed:.0%} of sessions failed (limit 5 %)"),
(mismatch > 0.001, f"re-run differs on {mismatch:.2%}"
" (limit 0.1 %)"),
(unblinded, "labels revealed before lock")):
if bad:
return f"D = {d:+.2f}: NOT INTERPRETABLE - {why}"
if lo >= SESOI: v = "X's Sharpe ratio is higher by at least the SESOI"
elif lo > 0: v = "X's Sharpe ratio is higher; not shown to reach the SESOI"
elif hi < 0: v = "Y's Sharpe ratio is higher"
elif lo >= -SESOI and hi <= SESOI: v = "no difference worth acting on"
else: v = "unclear: the interval is too wide to decide"
if mean_ratio < 0.95:
v += f"\n qualifier: X's mean P&L is {1 - mean_ratio:.0%} below Y's mean P&L"
return f"D = {d:+.2f} ({lo:+.2f} to {hi:+.2f}): {v}"
for case in ((5.10, 4.80, 5.40, 1.02), (0.40, 0.12, 0.68, 1.00),
(0.05, -0.20, 0.30, 0.99), (-0.50, -0.80, -0.20, 1.05),
(0.20, -0.45, 0.85, 1.00), (1.20, 0.90, 1.50, 0.88),
(2.00, 1.70, 2.30, 1.00, 0.08), (2.00, 1.70, 2.30, 1.00, 0.01, 0.004)):
print(verdict(*case))
print("secondary outcomes (mean P&L, P&L sd, max |inventory|, intra-session drawdown,")
print(" P&L inside pulses, fill rates) are reported with intervals; they never decide")
print(" the verdict")
# D = +5.10 (+4.80 to +5.40): X's Sharpe ratio is higher by at least the SESOI
# D = +0.40 (+0.12 to +0.68): X's Sharpe ratio is higher; not shown to reach the SESOI
# D = +0.05 (-0.20 to +0.30): no difference worth acting on
# D = -0.50 (-0.80 to -0.20): Y's Sharpe ratio is higher
# D = +0.20 (-0.45 to +0.85): unclear: the interval is too wide to decide
# D = +1.20 (+0.90 to +1.50): X's Sharpe ratio is higher by at least the SESOI
# qualifier: X's mean P&L is 12% below Y's mean P&L
# D = +2.00: NOT INTERPRETABLE - 8% of sessions failed (limit 5 %)
# D = +2.00: NOT INTERPRETABLE - re-run differs on 0.40% (limit 0.1 %)
# secondary outcomes (mean P&L, P&L sd, max |inventory|, intra-session drawdown,
# P&L inside pulses, fill rates) are reported with intervals; they never decide
# the verdictThe example results are made up. If any precondition fails (more than 5 % of sessions failed or truncated, the independent rerun disagreeing with the registered code’s results on more than 0.1 % of sessions, or the labels revealed before the results are locked), the report says “not interpretable” whatever the primary outcome. Only when the lower bound of the confidence interval is at least 0.31 does the report say “X’s Sharpe ratio is higher by at least the smallest effect of interest”; when the lower bound is above 0 but below 0.31, “X’s is higher, but the size is not established”; when the upper bound is below 0, “Y’s is higher”; when the interval lies within ±0.31, “no difference worth changing practice for”; otherwise, “cannot decide”. The qualifier does not change the category of the verdict but is appended to it: in the sixth example, for instance, X’s mean net value is 12 % below Y’s, and the report must say so.
What this experiment cannot show
First, the simulator is not a real market: the conclusions hold only for the simulator’s specification of noise traders, other market makers and informed agents, and real informed trading may be more hidden and more persistent, or may interact with inventory skewing. Second, the design of the toxic pulses determines the result; the protocol registers their distribution but cannot cover every form of adverse selection. Third, it tests one set of parameters (, one session length, one spread-matching method) and cannot be generalised to all inventory-control rules; nor does it test the exact solution of Chapter 2. Fourth, the Sharpe ratio is not the Avellaneda–Stoikov objective, and a higher Sharpe ratio does not mean higher expected utility. Fifth, latency, queue competition and fees are simplified in the simulator, while in real markets they often decide whether market making makes or loses money. Sixth, it does not test the Hawkes models of Chapter 3, the VPIN of Chapter 4 or the imbalance measures of Chapter 5; those need their own tests. Seventh, the large effect in the pilot environment comes from the author’s own model, which is exactly why an independent simulator and pre-registration are needed; if the independent experiment finds “no difference worth changing practice for”, that is the experiment’s conclusion.
Ethics and conflicts of interest
The experiment involves no human participants and needs no ethics committee approval, and it involves no real trading or money; but the protocol, simulator configuration, pulse-generating script, strategy code and analysis code must be registered publicly before any run. Whatever the result, positive or negative, the results, the code and summary data for all sessions will be published. To limit the influence of the author’s interests, the protocol and code are registered publicly before the third party runs anything, the author has no access to the seeds or the pulse schedule, and the results will be published whatever the outcome.
Disclosure
The author uses the works of 5D Library, including this book, as the methodological basis of his community courses, the 5D Ascension Cards (Dimensional Shift Card Decks) and the Yafu AI knowledge base, and therefore has a commercial interest in the subject of this book. The book promotes no course, card deck or service; the Chapter 8 trial is designed to be run and analysed by researchers independent of the author’s business and does not recruit from the author’s community, students or customers; all the code and the study protocol are published openly for anyone to check. The author develops and maintains quantitative research scripts and provides related open-source research and independent consulting services; while writing this book he may have held routine observation positions in US equity index funds (such as SPY) or related derivatives. All models in this book are offered solely for mathematical and historical-statistical discussion; they constitute no investment advice or trading signal, and the book recommends no specific trade or asset.
Chapter summary
The four-step checklist examines a trading day from four sides: adverse selection, order flow, inventory and execution; the example shows that adverse selection is visible only in realized spreads split by direction, and that inventory skewing can hide it in the totals. The pre-registered simulated-matching experiment compares the cross-session Sharpe ratios of inventory-skewed and fixed symmetric quotes under toxic pulses, in 8,600 paired sessions on a simulator frozen by an independent third party, with a single primary outcome (their difference), a smallest effect of interest of 0.31, double-blind running, a decision rule written as code and a pre-specified “lower mean return” qualifier; it will be published whatever the result. It tests one quoting rule in one simulator, cannot prove the book’s other models and cannot be carried over directly to real markets.
References
Fan, Y. (2026). High-Frequency Limit Order Book Microstructure, Hawkes Self-Exciting Point Processes and Adverse-Selection Flow Toxicity: From the Avellaneda–Stoikov Market-Making Equations to VPIN, Read Beside Classical Chinese Thought on Market Equilibrium [高频限价订单簿微观结构、霍克斯自激点过程与逆向选择毒流] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23828065 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).