5D Library › High-Frequency Limit Order Book Microstructure, Hawkes Self-Exciting Point Processes and Adverse-Selection Flow Toxicity · 中文

Chapter 5 Book Imbalance and Queues: Order Flow Imbalance, Queue Imbalance and Queue Position

The basic events of a limit order book

In a limit order book, orders queue at the best bid and best ask and execute by price priority and then time priority. Only three kinds of event change the top of the book: a new limit order joins a queue, an existing limit order is cancelled, or a market order consumes orders from the front of a queue. Cont, Stoikov and Talreja wrote these three kinds of event as a continuous-time Markov model and used it to compute conditional probabilities of events such as price changes and queue depletion (Cont et al. 2010); the monograph of Bouchaud and co-authors discusses such models and the empirical regularities systematically (Bouchaud et al. 2018). This chapter discusses two quantities that are easily confused: order flow imbalance (the net effect of book events over an interval) and queue imbalance (the relative sizes of the two queues at a moment).

Order flow imbalance and contemporaneous price changes

Evidence

Cont, Kukanov and Stoikov defined the contribution of each event nn at the best quotes as (Cont et al. 2014)

en=𝟙{PnB≥Pn−1B}qnB−𝟙{PnB≤Pn−1B}qn−1B−𝟙{PnA≤Pn−1A}qnA+𝟙{PnA≥Pn−1A}qn−1A,e_n = \mathbb{1}\{P^B_n \ge P^B_{n-1}\}\,q^B_n – \mathbb{1}\{P^B_n \le P^B_{n-1}\}\,q^B_{n-1} – \mathbb{1}\{P^A_n \le P^A_{n-1}\}\,q^A_n + \mathbb{1}\{P^A_n \ge P^A_{n-1}\}\,q^A_{n-1},

where PB,qBP^B, q^B and PA,qAP^A, q^A are the best bid and ask and their queued quantities. Increases on the bid and decreases on the ask count as positive, the reverse as negative; the sum of ene_n over an interval is the order flow imbalance (OFI). Using 50 randomly chosen S&P 500 stocks in NYSE TAQ data for April 2010, they regressed ΔPk=α+βOFIk+εk\Delta P_k = \alpha + \beta\,\mathrm{OFI}_k + \varepsilon_k over 10-second intervals (ΔP\Delta P in ticks) and found an average R2R^2 of 65 %; with trade imbalance alone (buyer-initiated minus seller-initiated volume) the average R2R^2 was 32 %, and with both included the explanatory power of trade imbalance essentially disappeared. The slope can be written βi=c/ADiλ\beta_i = c/AD_i^{\lambda}, with ADiAD_i the average depth; the estimates of λ\lambda averaged about 0.98, and λ=1\lambda = 1 could not be rejected for 35 of the 50 stocks, so price impact is roughly inversely proportional to depth. They also pointed out that under certain scaling assumptions OFI is approximately a noisy multiple of the square root of traded volume, so a “square-root” relation between price and volume can arise as a statistical consequence of aggregation, and is noisier and less robust than the relation based on OFI.

The author’s framework describes this research as “a linear relation between order book imbalance and short-term mid-price moves”. After checking, two points must be kept apart. First, CKS studied order flow imbalance, a flow over an interval, not a ratio of stocks of queued quantities. Second, it describes price changes within the same interval, a contemporaneous “explanation”, not a prediction of future prices. As for the square-root impact law for metaorders, Book 39 discussed its evidence and scope; here it is only recapped.

Queue imbalance and the next price move

The static queue imbalance I=(qB−qA)/(qB+qA)I = (q^B – q^A)/(q^B + q^A) is related to the direction of the next mid-price move: the relatively thicker the bid queue, the more likely the next move is up.

Evidence

The model of Cont and de Larrard treats the queues at the best bid and ask as two independent birth–death processes: limit orders arrive with intensity λ\lambda, cancellations occur with intensity θ\theta and market orders arrive with intensity μ\mu; the spread is always one tick, and when a queue empties the price moves one tick and both queues are redrawn from a fixed distribution (Cont and Larrard 2013). In this model the probability that the next price move is up is the probability that the ask queue empties before the bid queue, and it depends on the absolute sizes of the two queues. Gould and Bonart, using 10 liquid Nasdaq stocks in 2014 (LOBSTER data, 252 trading days), found that queue imbalance has significant predictive power for the direction of the next mid-price move, especially for large-tick stocks (whose spread is often one tick and whose queues are long) (Gould and Bonart 2015).

The program below implements a one-level book of the Cont–de Larrard kind: at each queue, limit orders, cancellations and market orders arrive at rates 1.2, 0.6 and 0.8 per second; when a queue empties the price moves one tick and both queues are redrawn uniformly from 1 to 2D−12D-1 (mean depth DD). The first part runs the OFI and trade-imbalance (TI) regressions over 10-second intervals; the second computes exactly, by Gauss–Seidel iteration, the probability that the next move is up when the bid queue is xx and the ask queue yy, and compares it with frequencies in the simulation grouped by imbalance.

Calculation

# A level-1 order book in the spirit of Cont & de Larrard (2013): unit orders; at each
# best queue, limit orders arrive at rate LAM, cancellations at rate TH (while the queue
# is non-empty) and market orders at rate MU. When a queue empties, the price moves one
# tick (spread stays one tick); both queues are then redrawn uniformly from 1..2D-1.
# Part 1: order flow imbalance (OFI) of Cont, Kukanov & Stoikov (2014) per 10-second
# interval, regression of mid-price changes (ticks) on OFI and on trade imbalance (TI),
# for two depths D. Part 2: probability that the next price move is up, given the two
# queue sizes, computed exactly (two competing birth-death queues) and compared with the
# queue imbalance I = (qb - qa) / (qb + qa).
import math, random, statistics

LAM, TH, MU = 1.2, 0.6, 0.8

def simulate(rng, D, T):
    t, pb, qb, qa = 0.0, 0, D, D; rate = 2 * (LAM + TH + MU); out = []
    while t < T:
        t += rng.expovariate(rate); u = rng.random() * rate
        side, kind = ("b", "a")[u >= rate / 2], u % (rate / 2)
        kind = "L" if kind < LAM else ("C" if kind < LAM + TH else "M")
        old = (pb, qb, pb + 1, qa)
        if side == "b":
            if kind == "L":
                qb += 1
            elif qb > 0 and (kind == "M" or kind == "C"):
                qb -= 1
        else:
            if kind == "L":
                qa += 1
            elif qa > 0 and (kind == "M" or kind == "C"):
                qa -= 1
        tr = (1 if side == "a" else -1) if kind == "M" else 0  # market buy hits the ask
        if qb == 0:
            pb -= 1; qb, qa = rng.randint(1, 2 * D - 1), rng.randint(1, 2 * D - 1)
        elif qa == 0:
            pb += 1; qb, qa = rng.randint(1, 2 * D - 1), rng.randint(1, 2 * D - 1)
        new = (pb, qb, pb + 1, qa)
        e = ((new[0] >= old[0]) * new[1] - (new[0] <= old[0]) * old[1]
             - (new[2] <= old[2]) * new[3] + (new[2] >= old[2]) * old[3])
        out.append((t, e, tr, pb, qb, qa))
    return out

def regress(x, y):
    mx, my = statistics.fmean(x), statistics.fmean(y)
    sxy = sum((a - mx) * (b - my) for a, b in zip(x, y))
    sxx = sum((a - mx) ** 2 for a in x)
    b = sxy / sxx; r2 = sxy * sxy / sxx / sum((v - my) ** 2 for v in y)
    return b, r2

def intervals(ev, h=10.0):
    ofi, ti, dp, k, cur, pb_last = [], [], [], 0, [0, 0, 0], 0
    for t, e, tr, pb, qb, qa in ev:
        while t > (k + 1) * h:
            ofi.append(cur[0]); ti.append(cur[1]); dp.append(pb_last - cur[2])
            cur = [0, 0, pb_last]; k += 1
        cur[0] += e; cur[1] += tr; pb_last = pb
    return ofi, ti, dp

def p_up(N=80):
    """P(ask queue empties before bid queue) for queue sizes (x, y), x, y < N."""
    up, dn = LAM, TH + MU; u = [[0.0] * (N + 1) for _ in range(N + 1)]
    for y in range(1, N + 1):
        u[0][y] = 0.0
    for x in range(1, N + 1):
        u[x][0] = 1.0
    for _ in range(3000):                              # Gauss-Seidel sweeps
        for x in range(1, N + 1):
            for y in range(1, N + 1):
                xp, yp = min(x + 1, N), min(y + 1, N)
                u[x][y] = (up * u[xp][y] + dn * u[x - 1][y] + up * u[x][yp]
                           + dn * u[x][y - 1]) / (2 * (up + dn))
    return u

if __name__ == "__main__":
    rng = random.Random(2014)
    print(f"rates per queue: limit {LAM}, cancel {TH}, market {MU}; 10-second intervals")
    print("   D    price moves   OFI: beta    R^2     TI: beta    R^2    beta x D")
    betas = []
    for D in (5, 20):
        ev = simulate(rng, D, 300_000.0)
        ofi, ti, dp = intervals(ev)
        b, r2 = regress(ofi, dp); bt, r2t = regress(ti, dp)
        moves = sum(abs(a) for a in dp); betas.append(b)
        print(f"  {D:3d}    {moves:9,d}      {b:.4f}   {r2:.3f}      {bt:.4f}   {r2t:.3f}"
              f"     {b * D:.3f}")
    print(f"   implied exponent in beta ~ D^-lambda: lambda ="
          f" {math.log(betas[0] / betas[1]) / math.log(4):.2f}")
    u = p_up()
    print("exact P(next move up | bid queue x, ask queue y):")
    for x, y in ((3, 1), (6, 2), (30, 10), (1, 1), (10, 10), (1, 3), (10, 30)):
        I = (x - y) / (x + y)
        print(f"   x {x:2d}, y {y:2d}: imbalance {I:+.2f}  P(up) {u[x][y]:.3f}")
    ev = simulate(rng, 5, 300_000.0)
    bins, pend, nxt, prev = {}, None, 10.0, ev[0]
    for t, e, tr, pb, qb, qa in ev:                       # sample the book every 10 s,
        if pend is not None and pb != pend[1]:            # then wait for the next move
            bins.setdefault(pend[0], []).append(pb > pend[1]); pend = None
        if t >= nxt and pend is None:
            I = (prev[4] - prev[5]) / (prev[4] + prev[5])
            pend = (min(4, int((I + 1) / 0.4)), prev[3]); nxt = t + 10.0
        prev = (t, e, tr, pb, qb, qa)
    print("simulated (D = 5): share of up-moves by queue imbalance sampled every 10 s")
    for k in sorted(bins):
        v = bins[k]
        lo = -1 + 0.4 * k
        print(f"   I in [{lo:+.1f}, {lo + 0.4:+.1f}):  {statistics.fmean(v):.3f}"
              f"  ({len(v):,} moves)")
# rates per queue: limit 1.2, cancel 0.6, market 0.8; 10-second intervals
#    D    price moves   OFI: beta    R^2     TI: beta    R^2    beta x D
#     5       19,819      0.0779   0.711      0.0901   0.114     0.389
#    20        5,217      0.0254   0.644      0.0187   0.028     0.508
#    implied exponent in beta ~ D^-lambda: lambda = 0.81
# exact P(next move up | bid queue x, ask queue y):
#    x  3, y  1: imbalance +0.50  P(up) 0.802
#    x  6, y  2: imbalance +0.50  P(up) 0.818
#    x 30, y 10: imbalance +0.50  P(up) 0.885
#    x  1, y  1: imbalance +0.00  P(up) 0.500
#    x 10, y 10: imbalance +0.00  P(up) 0.500
#    x  1, y  3: imbalance -0.50  P(up) 0.198
#    x 10, y 30: imbalance -0.50  P(up) 0.115
# simulated (D = 5): share of up-moves by queue imbalance sampled every 10 s
#    I in [-1.0, -0.6):  0.034  (3,000 moves)
#    I in [-0.6, -0.2):  0.212  (4,307 moves)
#    I in [-0.2, +0.2):  0.509  (6,621 moves)
#    I in [+0.2, +0.6):  0.775  (3,774 moves)
#    I in [+0.6, +1.0):  0.958  (3,464 moves)

In this model the R2R^2 of OFI on contemporaneous price changes is 0.711 (D=5D = 5) and 0.644 (D=20D = 20), against only 0.114 and 0.028 for trade imbalance, qualitatively in line with the CKS evidence. As depth increases from 5 to 20, the OFI slope falls from 0.0779 to 0.0254; the implied exponent λ=0.81\lambda = 0.81 is smaller than the CKS estimate of about 1, so “slope inversely proportional to depth” holds only approximately in this simplified model.

The second part shows that the imbalance on its own is not a sufficient statistic. At the same imbalance of +0.50, the probability of an up-move is 0.802 with queues (3, 1), 0.818 with (6, 2) and 0.885 with (30, 10); at −0.50 it is 0.198 with (1, 3) and 0.115 with (10, 30). When both sides are thicker, the “extra” quantity takes longer to be eaten, and the directional signal is stronger. In the simulation, the share of up-moves grouped by imbalance rises monotonically from 0.034 (I<−0.6I < -0.6) to 0.958 (I≥0.6I \ge 0.6), with 0.509 in the middle group. The relation is non-linear (S-shaped); “a linear relation” in the author’s framework applies only to OFI and contemporaneous price changes.

Queue position and adverse selection

Where an order sits in the queue also carries information. A buy order at the front of the queue fills quickly; one at the back fills only after everything ahead of it has been consumed, which tends to happen when selling pressure is heavy, the queue is about to empty and the price is about to fall. The program below is the author’s model: in the one-level book above, our buy order sits at position kk of 10 in the bid queue; new orders join behind us, cancellations remove other orders at random, and market sells eat from the front. If the ask queue empties first, the price ticks up and our order is left unfilled; if we are filled, we wait for the next mid-price change: an up-move gains 1.5 ticks relative to our purchase price, a down-move loses 0.5 tick.

Calculation

# Queue position and adverse selection of a resting buy order (author's toy model on the
# level-1 dynamics of p10: limit orders 1.2, cancellations 0.6, market orders 0.8 per
# second per side, unit orders, one-tick spread). Our order sits at position k in a bid
# queue of 10; new orders join behind it; a cancellation removes a random other order;
# market sells consume the queue from the front. Outcomes: the ask queue empties first
# (price ticks up, our order is left behind) or our order is filled. After a fill we wait
# for the next mid change: up -> +1.5 ticks against our price, down -> -0.5 ticks.
import random, statistics

LAM, TH, MU = 1.2, 0.6, 0.8

def one(rng, k, Q=10, qa0=10):
    ahead, behind, qa = k - 1, Q - k, qa0; filled = False
    R = 2 * (LAM + TH + MU)
    while True:
        u = rng.random() * R
        if u < LAM:                                    # bid side: a new limit order
            behind += 1
        elif u < LAM + TH:                             # a cancellation (not ours)
            n = ahead + behind
            if n > 0:
                if rng.random() * n < ahead:
                    ahead -= 1
                else:
                    behind -= 1
        elif u < LAM + TH + MU:                        # market sell hits the bid
            if not filled:
                if ahead > 0:
                    ahead -= 1
                else:
                    filled = True
            elif behind > 0:
                behind -= 1
        else:                                          # ask side
            v = u - (LAM + TH + MU)
            if v < LAM:
                qa += 1
            elif qa > 0:
                qa -= 1
        if qa == 0:
            return ("up", filled)                      # price ticks up
        if filled and ahead + behind == 0:
            return ("down", True)                      # bid queue gone after our fill

if __name__ == "__main__":
    rng = random.Random(2016)
    print("bid queue 10, ask queue 10; 40,000 orders per row")
    print("  position  P(filled)  P(next move down | filled)  P&L: per fill  per order")
    for k in (1, 3, 5, 8, 10):
        res = [one(rng, k) for _ in range(40_000)]
        fills = [r for r in res if r[1]]
        pdn = statistics.fmean(r[0] == "down" for r in fills)
        pnl = 1.5 * (1 - pdn) - 0.5 * pdn
        print(f"     {k:2d}       {len(fills) / len(res):.3f}              {pdn:.3f}"
              f"                 {pnl:+.3f}       {pnl * len(fills) / len(res):+.3f}")
    print("(no adverse selection: P(down | filled) = 0.5 and +0.5 tick per fill)")
# bid queue 10, ask queue 10; 40,000 orders per row
#   position  P(filled)  P(next move down | filled)  P&L: per fill  per order
#       1       0.999              0.500                 +0.500       +0.500
#       3       0.991              0.505                 +0.489       +0.485
#       5       0.972              0.515                 +0.469       +0.456
#       8       0.934              0.536                 +0.429       +0.401
#      10       0.905              0.551                 +0.398       +0.360
# (no adverse selection: P(down | filled) = 0.5 and +0.5 tick per fill)

A buy order in position 1 is almost certain to fill (0.999), and after the fill the next move is down with probability 0.500: there is no adverse selection, and each fill earns half the spread on average (+0.500 tick). In position 10 the fill probability falls to 0.905, the probability of a fall after the fill rises to 0.551, and the mean gain per fill falls to +0.398 tick, or +0.360 tick per order placed. The further back in the queue, the more “conditional” the fill: the very selling pressure that fills us also foreshadows a fall in price. This is Chapter 1’s adverse selection at the level of the queue, and it shows why queue position itself has value in a price–time priority market.

Note

On iceberg orders. The author’s framework mentions “iceberg-order splitting” in Chapter 6. An iceberg order is a limit order that displays only part of its quantity, replenishing the displayed part after it is filled; under price–time priority the replenished part usually goes to the back of the queue, so it trades queue position for less information exposure. Esser and Mönch gave a model of the optimal displayed size balancing fill probability against exposure costs (Esser and Mönch 2007). This is a question about how a limit order is displayed, different from the Almgren–Chriss schedule for splitting a parent order in Chapter 6.

Hypothesis · not established · The absolute size of imbalance

The author’s hypothesis (not established): for large-tick stocks, predicting the direction of the next mid-price move from the absolute sizes of the two queues (not only their imbalance ratio) gives a lower out-of-sample log loss than a model using the imbalance ratio alone. What would show it wrong: with stocks, periods and model specifications fixed in advance, a confidence interval for the difference in out-of-sample log loss between the two models that includes zero.

Chapter summary

Order flow imbalance (OFI) is the net effect of events at the best quotes over an interval; it is close to linear in contemporaneous mid-price changes, with an average R2R^2 of 65 % in CKS and a slope roughly inversely proportional to depth; it is not queue imbalance. Queue imbalance is related to the direction of the next price move, but non-linearly, and the relation depends on the absolute sizes of the two queues. Queue position determines how adversely selected a fill is: in the author’s model a buy order at the back of the queue sees the price fall after its fill with probability 0.551, and earns about 20 % less on average than one at the front.

References

Bouchaud, Jean-Philippe, Julius Bonart, Jonathan Donier, and Martin Gould. 2018. Trades, Quotes and Prices: Financial Markets Under the Microscope. Cambridge University Press. https://doi.org/10.1017/9781316659335.
Cont, Rama, Arseniy Kukanov, and Sasha Stoikov. 2014. “The Price Impact of Order Book Events.” Journal of Financial Econometrics 12 (1): 47–88.
Cont, Rama, and Adrien de Larrard. 2013. “Price Dynamics in a Markovian Limit Order Market.” SIAM Journal on Financial Mathematics 4 (1): 1–25.
Cont, Rama, Sasha Stoikov, and Rishi Talreja. 2010. “A Stochastic Model for Order Book Dynamics.” Operations Research 58 (3): 549–63. https://doi.org/10.1287/opre.1090.0780.
Esser, Angelika, and Burkart Mönch. 2007. “The Navigation of an Iceberg: The Optimal Use of Hidden Orders.” Finance Research Letters 4 (2): 68–81.
Gould, Martin D., and Julius Bonart. 2015. Queue Imbalance as a One-Tick-Ahead Price Predictor in a Limit Order Book. arXiv preprint arXiv:1512.03492.

Fan, Y. (2026). High-Frequency Limit Order Book Microstructure, Hawkes Self-Exciting Point Processes and Adverse-Selection Flow Toxicity: From the Avellaneda–Stoikov Market-Making Equations to VPIN, Read Beside Classical Chinese Thought on Market Equilibrium [高频限价订单簿微观结构、霍克斯自激点过程与逆向选择毒流] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23828065 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).