5D Library › Liquidation Cascades, Order-Book Depth Depletion and Cointegrated Statistical Arbitrage · 中文
Chapter 5 Cointegration, Pairs Trading and Broken Relationships
Spurious regression
Regress one of two completely unrelated random walks on the other and you will often get a highly “significant” slope and a high goodness of fit. Granger and Newbold demonstrated this spurious regression by simulation in 1974 (Granger and Newbold 1974): the ordinary regression test assumes stationary residuals, but a linear combination of two random walks is in general still a random walk, so the nominal level of the test collapses. That two prices “look alike” says nothing statistically.
Cointegration and the two-step method
Engle and Granger gave the positive definition in 1987 (Engle and Granger 1987): two series that are each integrated of order one (non-stationary like a random walk) are cointegrated if some linear combination is stationary; is the cointegrating coefficient (in pairs trading, the hedge ratio). Their two-step method:
- Regress by ordinary least squares and take the residuals ;
- Test the residuals for a unit root: , using the statistic of to test “the residuals have a unit root (no cointegration)”.
The second step is Dickey and Fuller’s 1979 test (Dickey and Fuller 1979), called the augmented Dickey–Fuller (ADF) test when lagged differences are added (Said and Dickey 1984). But there is a trap: the least squares in step one picks precisely the combination with the smallest residual variance, the one that looks most stationary, so the residual statistic leans further negative than in an ordinary unit-root test, and ordinary Dickey–Fuller critical values cannot be used. Phillips and Ouliaris derived the asymptotic distributions of such residual-based tests in 1990 (Phillips and Ouliaris 1990), and MacKinnon tabulated critical values corrected for sample size (MacKinnon 1991, 2010).
Evidence
Critical values (MacKinnon 2010, Table 2, with constant, no trend). One variable (ordinary unit-root test): −3.430 at 1 %, −2.862 at 5 %, −2.567 at 10 % (asymptotic). Two variables (Engle–Granger residual test): −3.896 at 1 %, −3.336 at 5 %, −3.044 at 10 %. The finite-sample correction is ; for two variables at 5 %, and , so with the critical value is about −3.361 (MacKinnon 2010).
Apply the Engle–Granger test to two independent random walks (250 observations each), 20,000 times, and compare the share rejecting “no cointegration” at three critical values; then see how many “cointegrated pairs” a pairwise screen of 100 unrelated series (4,950 pairs) finds; finally, see how often one year of data detects pairs that really are cointegrated, with spread half-lives of 10 and 40 days.
Calculation
# Engle-Granger: OLS of y on x, then ADF (one lag) on the residuals. Residual critical
# values (MacKinnon 2010, N = 2, constant): 5 % -3.336; T = 250: -3.361.
import math, random
rng = random.Random(1987)
T = 250
def ols(ys, xs): # y = a + b x
n = len(xs); mx, my = sum(xs) / n, sum(ys) / n
sxy = sum((x - mx) * (y - my) for x, y in zip(xs, ys))
b = sxy / sum((x - mx) ** 2 for x in xs)
return my - b * mx, b
def adf_t(e): # regression without constant, one lag of de
d = [e[i] - e[i - 1] for i in range(1, len(e))]
Y, X1, X2 = d[1:], e[1:-1], d[:-1]
s11 = sum(a * a for a in X1); s22 = sum(a * a for a in X2)
s12 = sum(a * b for a, b in zip(X1, X2))
s1y = sum(a * y for a, y in zip(X1, Y)); s2y = sum(a * y for a, y in zip(X2, Y))
det = s11 * s22 - s12 * s12
g = (s22 * s1y - s12 * s2y) / det; c = (s11 * s2y - s12 * s1y) / det
rss = sum((y - g * a - c * b) ** 2 for y, a, b in zip(Y, X1, X2))
return g / math.sqrt(rss / (len(Y) - 2) * s22 / det)
def eg(y, x):
a, b = ols(y, x)
return adf_t([v - a - b * u for v, u in zip(y, x)])
def walk(n):
w = [0.0]
for _ in range(n - 1): w.append(w[-1] + rng.gauss(0, 1))
return w
R = 20000
null = sorted(eg(walk(T), walk(T)) for _ in range(R))
print(f"two independent random walks, T = {T}, {R:,} replications")
for name, cv in (("Dickey-Fuller -2.86 (wrong table)", -2.86),
("MacKinnon asymptotic -3.336", -3.336),
("MacKinnon T = 250 -3.361", -3.361)):
rate = sum(t < cv for t in null) / R
print(f" reject 'no cointegration' at {name:34s}: {rate:5.1%}")
print(f" simulated 5 % critical value: {null[R // 20]:.3f}")
counts = []
for _ in range(4): # four universes of 100 unrelated series
w = [walk(T) for _ in range(100)]
counts.append(sum(eg(w[i], w[j]) < -3.361
for i in range(100) for j in range(i + 1, 100)))
print(f"screening 100 unrelated series (4,950 pairs), four universes:")
print(f" {counts} pairs 'cointegrated' at 5 %")
for hl in (10, 40):
b = 0.5 ** (1 / hl); power = 0
for _ in range(1000):
x = walk(T); s = [rng.gauss(0, 1 / math.sqrt(1 - b * b))]
for _ in range(T - 1): s.append(b * s[-1] + rng.gauss(0, 1))
power += eg([u + v for u, v in zip(x, s)], x) < -3.361
print(f"truly cointegrated, spread half-life {hl} days: detected in"
f" {power / 1000:.1%}")
# two independent random walks, T = 250, 20,000 replications
# reject 'no cointegration' at Dickey-Fuller -2.86 (wrong table) : 14.8%
# reject 'no cointegration' at MacKinnon asymptotic -3.336 : 5.0%
# reject 'no cointegration' at MacKinnon T = 250 -3.361 : 4.7%
# simulated 5 % critical value: -3.339
# screening 100 unrelated series (4,950 pairs), four universes:
# [356, 224, 323, 351] pairs 'cointegrated' at 5 %
# truly cointegrated, spread half-life 10 days: detected in 49.5%
# truly cointegrated, spread half-life 40 days: detected in 8.7%The cost of the wrong table is concrete: at −2.86, 14.8 % of pairs of completely unrelated random walks are judged cointegrated, about three times the nominal level. With MacKinnon’s critical values the error rate returns to about 5 % (the simulated 5 % quantile, −3.339, is very close to the asymptotic −3.336).
Even with the right critical values, screening itself manufactures false pairs in bulk: testing every pair among 100 unrelated series, four simulations found 356, 224, 323 and 351 “cointegrated” pairs, about 6 % on average (at the 4.7 % false-positive rate the expected count is about 233). The pairs share series and are correlated, so the count from any one screen fluctuates widely, and even the average of four screens need not be close to the expected value. Conversely, power is low: a truly cointegrated pair whose spread has a 10-day half-life is detected from one year of data only 49.5 % of the time; with a 40-day half-life, 8.7 %. On one year of daily data, “passing a cointegration test” and “being cointegrated” are only loosely related.
The evidence on pairs trading
Evidence
Pairs-trading returns and their decay. Using daily US stock data from 1962 to 2002, Gatev, Goetzmann and Rouwenhorst matched stocks by the minimum distance between normalised prices (the distance method, not a cointegration test) and found that a simple trading rule earned average annualised excess returns of up to 11 % for self-financing portfolios of pairs, typically exceeding conservative estimates of transaction costs (Gatev et al. 2006). Do and Faff extended the sample to June 2009 and found that profitability kept falling: the mean monthly excess return of the top 20 pairs dropped from 0.86 % in 1962–1988 to 0.37 % in 1989–2002 and 0.24 % in 2003–2009; yet the strategy did well in prolonged turbulence such as the 2000–2002 bear market and the 2007–2009 financial crisis; they attributed the decline mainly to worsening arbitrage risks (up to about 70 % of the drop in profits) rather than to hedge-fund competition (Do and Faff 2010). Their follow-up, allowing for commissions, market impact and short-selling fees, found that well-matched pairs within refined industry groups still earned a risk-adjusted return of about 30 basis points a month, 24 basis points when restricted to the largest 30 % of stocks, and that after 2002 both pairs trading and industry-relative short-term reversal were largely unprofitable (Do and Faff 2012).
Evidence
Statistical arbitrage and crowding. Avellaneda and Lee built mean-reversion signals from the residuals of regressions on principal components or on sector exchange-traded funds (modelling the residuals as OU processes); after transaction costs, the PCA strategies had an average annual Sharpe ratio of 1.44 in 1997–2007, falling to 0.9 in 2003–2007; the ETF-based strategies had 1.1 in 1997–2007, and 1.51 in 2003–2007 when volume signals were added (Avellaneda and Lee 2010). Khandani and Lo analysed the “quant meltdown” of August 2007: a simple simulated market-making strategy made significant losses in the week of 6 August but was profitable before and after; they attributed the dislocation to the deleveraging of portfolios from July to the first week of August and a temporary withdrawal of market-making risk capital from 8 August (Khandani and Lo 2011). When crowded strategies of the same kind unwind together, that alone can break the mean reversion of spreads.
When the relationship breaks
Cointegration describes the past. A merger, a change in regulation, a business restructuring or simply fund flows can shift the mean of a once-stationary spread, or turn it into a random walk.
Use a synthetic spread (half-life 10 days, stationary standard deviation 1 %) to compare three trading years: A, a stable relationship; B, the spread’s mean jumps by 3 % on day 60 of the trading period (a re-pricing); C, from day 60 of the trading period the spread becomes a random walk (cointegration ends). Rule: estimate the mean and standard deviation over a 250-day formation period, open at , close when returns to 0; an optional stop closes at and stops trading the pair for the rest of the year. 1,000 simulations per case; costs of 0, 10 and 20 basis points per leg per transaction.
Calculation
# Pairs rule on a synthetic OU spread (half-life 10 d, sd 1 %); scenarios A stable,
# B mean +3 % on day 60, C random walk from day 60. Costs in bp per leg per trade.
import math, random
rng = random.Random(2006)
HL, SD = 10, 1.0
B = 0.5 ** (1 / HL); E = SD * math.sqrt(1 - B * B)
def year(scen):
s = [rng.gauss(0, SD)]
for t in range(499):
if scen == "C" and t >= 309: s.append(s[-1] + rng.gauss(0, E))
else:
mu = 3.0 if scen == "B" and t >= 309 else 0.0
s.append(mu + B * (s[-1] - mu) + rng.gauss(0, E))
return s
def trade(s, stop):
f = s[:250]; m = sum(f) / 250; sd = math.sqrt(sum((v - m) ** 2 for v in f) / 249)
pos, pnl, legs = 0, 0.0, 0
for t in range(250, 499):
z = (s[t] - m) / sd
if pos and stop and abs(z) > 4:
legs += 2; pos = 0; break
if pos == 0 and abs(z) > 2: pos = -1 if z > 0 else 1; legs += 2
elif pos and (z * pos >= 0): pos = 0; legs += 2
pnl += pos * (s[t + 1] - s[t])
if pos: legs += 2 # close at year end
return pnl, legs
print("scenario, stop | mean P&L (%) at cost 0 / 10 / 20 bp | P(loss) at 10 bp | 5th pct")
for scen in "ABC":
for stop in (False, True):
res = [trade(year(scen), stop) for _ in range(1000)]
nets = {c: sorted(p - l * c / 100 for p, l in res) for c in (0, 10, 20)}
mean = " ".join(f"{sum(v) / 1000:5.2f}" for v in nets.values())
loss = sum(v < 0 for v in nets[10]) / 1000
print(f" {scen}, {'stop 4' if stop else 'none '} | {mean}"
f" | {loss:5.1%} | {nets[10][50]:6.2f}")
# scenario, stop | mean P&L (%) at cost 0 / 10 / 20 bp | P(loss) at 10 bp | 5th pct
# A, none | 5.63 4.50 3.37 | 0.5% | 0.00
# A, stop 4 | 5.08 4.02 2.96 | 4.5% | 0.00
# B, none | 0.73 0.03 -0.67 | 52.4% | -2.82
# B, stop 4 | -0.19 -0.85 -1.51 | 76.8% | -2.65
# C, none | 1.22 0.25 -0.73 | 44.7% | -7.26
# C, stop 4 | 1.12 0.26 -0.59 | 57.5% | -2.71With a stable relationship the rule earns on average 4.50 % a year (on the notional of one leg) at 10 basis points per leg, and only 0.5 % of years lose money. When the mean jumps, the average return falls to almost zero and more than half the years lose; the stop makes things worse (an average of −0.85 %, 76.8 % of years losing), because the spread, oscillating around its new mean, keeps triggering it. When the spread becomes a random walk, the average return is also close to zero, 44.7 % of years lose, and the worst 5 % of years lose more than 7.26 %; the stop cuts the worst-5 % loss to about 2.71 % but does not improve the average. Costs matter as well: even with a stable relationship, raising costs from 0 to 20 basis points cuts the average return from 5.63 % to 3.37 %.
Note
A stop trims the tail; it does not improve the mean. In this synthetic example the stop limits the worst outcomes rather than raising the expected return, and when the spread merely moves to a new mean it does harm. Whether a stop rule is good depends on the form the break takes, which is not known in advance. That is why the checklist in Chapter 8 asks for the stop rule to be written down beforehand and for cointegration stability to be checked.
Chapter summary
Regressions between random walks produce spurious relationships; cointegration requires a stationary linear combination, and the Engle–Granger two-step test must use residual-based critical values (two variables with a constant, about −3.34 at 5 %, about −3.36 with 250 observations); using the ordinary Dickey–Fuller −2.86 pushes the error rate close to 15 %. Even with correct critical values, large-scale screening produces hundreds of false pairs, while one year of data has little power to detect genuine cointegration. Historical pairs-trading returns were real but have kept decaying and shrink sharply after costs. When a cointegrating relationship breaks, the rule’s returns go to zero or below, and stops only trim the tail. The next chapter takes a different approach: instead of assuming a fixed hedge ratio, let it change over time.
References
Fan, Y. (2026). Liquidation Cascades, Order-Book Depth Depletion and Cointegrated Statistical Arbitrage: Ornstein-Uhlenbeck Processes and Kalman-Filter Dynamic Hedging [高杠杆衍生品清算瀑布、订单簿深度衰竭与协整统计套利] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23796776 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).