5D Library › Phase Transitions, Critical Slowing Down and Resilience in Complex Dynamical Systems · 中文
Chapter 3 The Limits of Early Warning: Signal to Noise, Selection Bias and Flips Without Warning
Two questions
To be useful, a warning method must answer two questions at once: how often does it raise the alarm when the system really is approaching a threshold (the detection rate), and how often when it is not (the false-alarm rate)? Judged on the first question alone, any method that often cries wolf looks impressive. Chapter 6 of Volume 20 of 5D Library demonstrated a simple rule on autoregressive series (a warning whenever Kendall’s ); this chapter puts the question on a systematic footing in the grazing model and then turns to two more fundamental difficulties: transitions that leave no trace of slowing down, and the bias created by studying only systems that have already shifted.
ROC curves
Treat Kendall’s as a score and raise the alarm above some cut-off. As the cut-off moves down, detection and false-alarm rates rise together; plotting (false-alarm rate, detection rate) for every cut-off gives the ROC curve. The area under it, the AUC, equals the probability that a randomly chosen series that really approaches a threshold scores higher than a randomly chosen one that does not (Hanley and McNeil 1982): 0.5 is a coin toss, 1 perfect separation. For the general principles see Volume 26, Chapter 4.
Evidence
ROC evaluation of warning indicators. For a simulated data set, a chemostat experiment and a glaciation record, Boettiger and Hastings fitted both a stable model and a model approaching a saddle-node bifurcation, simulated 500 series from each fitted model, and compared warning indicators by ROC curves: the distributions of Kendall’s in the two groups overlapped heavily, and a stable simulated series could give by chance. With the false-alarm rate fixed at 5 %, the variance indicator detected only 25 %, 5.0 % and 5.4 % in the three data sets, against 61 %, 34 % and 100 % for their model-based likelihood method (Boettiger and Hastings 2012b).
Calculation
Grazing model (, 1,000 time units): in the “approaching” group the pressure rises from 1.0 to 2.5, stopping short of the fold at 2.604; in the “null” group it stays at 1.0. With 200 series per group, the same process is observed in three ways: once per time unit (1,000 points); once every 5 units (200 points); only the last 200 units (200 points). After detrending we compute Kendall’s for autocorrelation and variance in a moving window and report the AUC and the detection rate when false alarms are held to 5 %.
Calculation
# ROC of Kendall's tau of rolling AC(1), variance (Boettiger & Hastings 2012, J R Soc Interface).
# Grazing model, sigma = 0.15, 1,000 time units; "approaching": c rises 1.0 -> 2.5 (fold at
# 2.604); "null": c stays at 1.0. Three views of the same runs: every unit (1,000 points),
# every 5th unit (200 points), only the last 200 units (200 points). 200 series each.
import math, random
rng = random.Random(6)
def simulate(ramp, T=1000, sigma=0.15, dt=0.1):
x, out = 8.9, []
for t in range(T):
c = 1.0 + (1.5 * t / T if ramp else 0.0)
for _ in range(10):
x += ((x * (1 - x / 10) - c * x * x / (x * x + 1)) * dt
+ sigma * math.sqrt(dt) * rng.gauss(0, 1))
out.append(x)
return out
def detrend(y, bw):
k = [(d, math.exp(-0.5 * (d / bw) ** 2)) for d in range(-2 * bw, 2 * bw + 1)]
out = []
for i in range(len(y)):
ok = [(w, y[i + d]) for d, w in k if 0 <= i + d < len(y)]
out.append(y[i] - sum(w * u for w, u in ok) / sum(w for w, _ in ok))
return out
def stats(w):
m = sum(w) / len(w); v = sum((u - m) ** 2 for u in w)
return sum((w[i] - m) * (w[i + 1] - m) for i in range(len(w) - 1)) / v, v / len(w)
def kendall(y):
s = sum((y[j] > y[i]) - (y[j] < y[i]) for i in range(len(y)) for j in range(i + 1, len(y)))
return s / (len(y) * (len(y) - 1) / 2)
def taus(y):
r = detrend(y, max(3, len(y) // 40)); win = len(r) // 2; step = max(1, win // 25)
a, v = zip(*(stats(r[i - win:i]) for i in range(win, len(r) + 1, step)))
return kendall(a), kendall(v)
def auc(pos, neg): # P(score of an approaching series > score of a null one)
return sum((p > q) + 0.5 * (p == q) for p in pos for q in neg) / (len(pos) * len(neg))
def tpr_at(pos, neg, fpr=0.05):
thr = sorted(neg)[int(math.ceil((1 - fpr) * len(neg))) - 1]
return sum(p > thr for p in pos) / len(pos), thr
DESIGNS = (("every unit, 1,000 pts", lambda y: y), ("every 5th unit, 200 pts", lambda y: y[::5]),
("last 200 units, 200 pts", lambda y: y[-200:]))
def run(n=200):
series = {k: [simulate(k) for _ in range(n)] for k in (True, False)}
return {name: {k: [taus(view(y)) for y in v] for k, v in series.items()}
for name, view in DESIGNS}
if __name__ == "__main__":
res = run()
print("design indicator AUC detected at 5 % false alarms (tau cut-off)")
for name, _ in DESIGNS:
for j, lab in ((0, "AC(1)"), (1, "variance")):
pos = [t[j] for t in res[name][True]]; neg = [t[j] for t in res[name][False]]
tp, thr = tpr_at(pos, neg)
print(f"{name:25s} {lab:9s} {auc(pos, neg):.2f} {tp:4.0%}"
f" ({thr:+.2f})")
neg = [t[0] for t in res[DESIGNS[0][0]][False]]
print(f"null series with AC(1) tau > 0.5: {sum(t > 0.5 for t in neg) / len(neg):.0%}"
f"; largest null tau {max(neg):+.2f}")
# design indicator AUC detected at 5 % false alarms (tau cut-off)
# every unit, 1,000 pts AC(1) 0.94 66% (+0.82)
# every unit, 1,000 pts variance 0.98 86% (+0.79)
# every 5th unit, 200 pts AC(1) 0.66 16% (+0.74)
# every 5th unit, 200 pts variance 0.90 52% (+0.79)
# last 200 units, 200 pts AC(1) 0.54 5% (+0.82)
# last 200 units, 200 pts variance 0.56 8% (+0.77)
# null series with AC(1) tau > 0.5: 26%; largest null tau +0.90
Four points stand out. First, with dense records spanning the whole change, both indicators separate well: AUC 0.94 for autocorrelation and 0.98 for variance, detecting 66 % and 86 % at a 5 % false-alarm rate. Second, holding false alarms to 5 % requires a cut-off of about +0.8 for , not the customary +0.5: among the null series, where nothing changed, 26 % had an autocorrelation above 0.5 and the largest reached +0.90; statistics in overlapping moving windows are strongly dependent, so is far more widely spread than for independent data. Third, the density and the span of the record play different roles: with the same 200 points recorded every 5 units across the whole process, variance remains useful (AUC 0.90, 52 % detected) while autocorrelation drops sharply (0.66, 16 %), because the lag-1 autocorrelation between neighbouring points is itself small; with only the last 200 units, both are close to a coin toss (0.54 and 0.56). Dakos and colleagues likewise note that variance estimates benefit from more frequent observation while autocorrelation estimates need a longer span (Dakos et al. 2015). Fourth, these figures come from a model with known parameters, constant noise and a fold that really exists; real data can only be harder.
Flips without warning
Evidence
Noise-induced transitions. Ditlevsen and Johnsen examined whether variance and autocorrelation rose before the Dansgaard–Oeschger events (abrupt warmings within decades) in the NGRIP ice core from Greenland: neither increased significantly. They concluded that these jumps were more probably induced by noise than by an approaching bifurcation, and that hoping for early warning of them may be “wishful thinking” (Ditlevsen and Johnsen 2010). Using several ecological models, Hastings and Wysham showed that only a limited class of systems displays the usual leading indicators; many nonlinear ecological models under environmental variability have no smooth potential and give no forewarning of an abrupt change (Hastings and Wysham 2010).
Evidence
A list of failures. Dakos and colleagues listed many ways in which warnings fail: large process or observation error, drivers that change too fast, interacting thresholds, monitoring the wrong variable, non-local bifurcations; noise-induced transitions, long transients and large jumps in the driver involve no bifurcation at all (Dakos et al. 2015). Conversely, Kéfi and colleagues found the same indicators before non-catastrophic, smooth transitions, so a rising indicator shows only that a system is becoming more sensitive to disturbance, not what kind of transition is coming (Kéfi et al. 2013).
Selection bias: the prosecutor’s fallacy
Evidence
The prosecutor’s fallacy. Boettiger and Hastings simulated a population model with constant parameters, passing through no bifurcation and collapsing only by chance, selected the replicates that happened to collapse and analysed the windows before the collapse: Kendall’s for variance and autocorrelation was higher there than in randomly chosen simulations. They likened this reasoning to the prosecutor’s fallacy, confusing “the probability of collapse given a warning” with “the probability of a warning given a collapse”; truncating the data after the fact does not easily remove the bias, while replicated experiments avoid it (Boettiger and Hastings 2012a).
Calculation
The pressure is held constant. A: with stronger noise (), 400 series of 2,000 time units each; some cross the boundary by chance and collapse. We compare “the 400 units before a collapse” with “400 units cut at random from series that never collapsed”. B: with still stronger noise (), to watch the system flicker back and forth between the two basins.
Calculation
# Transitions without a warning, and warnings without an approach. Grazing model with the
# pressure c held CONSTANT (no bifurcation is approached; fold at 2.604).
# A. Chance (noise-induced) collapses, c = 2.3, sigma = 0.5, 2,000 time units per series.
# Indicators in the 400 units before each collapse are compared with 400-unit windows
# from series that did not collapse: selecting on the outcome (Boettiger & Hastings 2012).
# B. Flickering, c = 2.2, sigma = 1.0: the state jumps back and forth between the basins.
import math, random
rng = random.Random(21)
def run(c, sigma, T, x=None, dt=0.1, stop=True):
x = 6.6 if x is None else x
out = []
for t in range(T):
for _ in range(10):
x += ((x * (1 - x / 10) - c * x * x / (x * x + 1)) * dt
+ sigma * math.sqrt(dt) * rng.gauss(0, 1))
x = max(x, 0.01)
out.append(x)
if stop and x < 2.5: break
return out
def stats(w):
m = sum(w) / len(w); v = sum((u - m) ** 2 for u in w)
return sum((w[i] - m) * (w[i + 1] - m) for i in range(len(w) - 1)) / v, v / len(w)
def kendall(y):
s = sum((y[j] > y[i]) - (y[j] < y[i]) for i in range(len(y)) for j in range(i + 1, len(y)))
return s / (len(y) * (len(y) - 1) / 2)
def taus(seg, win=200, step=8): # no detrending needed: c is constant
a, v = zip(*(stats(seg[i - win:i]) for i in range(win, len(seg) + 1, step)))
return kendall(a), kendall(v)
if __name__ == "__main__":
tipped, calm = [], []
for _ in range(400):
y = run(2.3, 0.5, 2000)
if y[-1] < 2.5 and len(y) >= 420:
tipped.append(taus(y[-402:-2])) # ends 2 units before the fall
elif y[-1] >= 2.5:
s = rng.randrange(0, 1600); calm.append(taus(y[s:s + 400]))
print(f"A. 400 series: {len(tipped)} collapsed by chance after t >= 420,"
f" {len(calm)} never collapsed")
for lab, grp in (("windows before a chance collapse", tipped),
("windows from calm series", calm)):
ta = [t[0] for t in grp]; tv = [t[1] for t in grp]
print(f" {lab:33s} mean tau AC(1) {sum(ta) / len(ta):+.2f},"
f" variance {sum(tv) / len(tv):+.2f}")
print(f" {'':33s} share with tau > 0.5: AC(1) {sum(a > 0.5 for a in ta) / len(ta):.0%},"
f" variance {sum(b > 0.5 for b in tv) / len(tv):.0%}, both"
f" {sum(a > 0.5 and b > 0.5 for a, b in grp) / len(grp):.0%}")
y = run(2.2, 1.0, 4000, stop=False)
low = [u < 3.0 for u in y]
flips = sum(1 for u, v in zip(low, low[1:]) if u != v)
a, v = stats(y)
a0, v0 = stats(run(2.2, 0.15, 4000, stop=False))
print(f"B. c = 2.2, sigma = 1.0: {flips} crossings of x = 3 in 4,000 units;"
f" time in lower basin {sum(low) / len(low):.0%}")
print(f" AC(1) {a:.2f}, variance {v:.2f} (same c, sigma = 0.15: AC(1) {a0:.2f},"
f" variance {v0:.3f})")
hist = [0] * 10
for u in y: hist[min(9, int(u))] += 1
print(" histogram of x (bins of width 1): " + " ".join(f"{h / len(y):.0%}" for h in hist))
# A. 400 series: 87 collapsed by chance after t >= 420, 286 never collapsed
# windows before a chance collapse mean tau AC(1) +0.10, variance +0.18
# share with tau > 0.5: AC(1) 21%, variance 29%, both 17%
# windows from calm series mean tau AC(1) +0.01, variance +0.04
# share with tau > 0.5: AC(1) 20%, variance 24%, both 16%
# B. c = 2.2, sigma = 1.0: 140 crossings of x = 3 in 4,000 units; time in lower basin 16%
# AC(1) 0.91, variance 5.21 (same c, sigma = 0.15: AC(1) 0.68, variance 0.031)
# histogram of x (bins of width 1): 7% 5% 4% 5% 8% 17% 25% 20% 8% 1%A: of 400 series, 87 collapsed by chance after time 420 and 286 never collapsed. The parameters never changed, yet in the windows before a collapse the mean for autocorrelation and variance were +0.10 and +0.18, against +0.01 and +0.04 in calm series: merely by selecting the series that “later collapsed”, the indicators shift upwards. The bias is small (shares with above 0.5: 21 % against 20 % for autocorrelation, 29 % against 24 % for variance) but consistent in direction. Even among calm series, 16 % of windows had both values of above 0.5.
B: at a pressure of 2.2, far from the fold, strong noise makes the system cross 140 times in 4,000 units and spend 16 % of the time in the lower basin; autocorrelation is 0.91 and variance 5.21, against 0.68 and 0.031 with weak noise at the same pressure, and the histogram has two peaks. The indicators are “high”, not because a threshold is near, but because the noise is strong enough to push the system repeatedly across the boundary.
Evidence
Flickering. Dakos, van Nes and Scheffer pointed out that with strong noise a system may flicker between basins and shift while still far from a bifurcation, so that slowing-down indicators may fail; flickering also raises variance, autocorrelation and skewness, but the distribution of states of a flickering system can be used to map the alternative attractors (Dakos et al. 2013). Wang and colleagues analysed about 125 years of diatom records in sediment from Lake Erhai in Yunnan and reported flickering up to about 20 years before the lake’s transition to a eutrophic state (Wang et al. 2012).
Real data
Evidence
126 long-term ecological data sets. Burthe and colleagues used 126 long-term abundance data sets for 55 taxa from marine and freshwater ecosystems to test whether variance and autocorrelation predict nonlinear turning points in abundance. True positives were rare: 9 % for variance (16 of 170) and 13 % for autocorrelation (19 of 152); false positives were more common than false negatives (53 % against 38 % for variance, 47 % against 40 % for autocorrelation); and the two indicators disagreed about 43 % of the series. The authors concluded that the two indicators “consistently fail” to predict nonlinear change and recommended combining several indicators with analyses of human drivers (Burthe et al. 2016). The 2012 methods comparison by Dakos and colleagues also advised sensitivity analysis of the detrending bandwidth and window length and the combination of several indicators (Dakos et al. 2012).
Note
Using early warnings honestly. At a minimum: (1) write down the warning rule and cut-off in advance; (2) run the same rule on many series or periods without a transition and report the false-alarm rate and the ROC; (3) look at variance and autocorrelation together and check whether the noise itself is changing; (4) analyse the sensitivity to window and detrending bandwidth; (5) state which kinds of transition the indicators cannot detect. A rising indicator is best read as “the system is becoming more sensitive to disturbance, and the cause is worth finding”, not as “the threshold will arrive on such a date”. The checklist in Chapter 8 uses it in this way.
Chapter summary
Warning indicators must be judged by detection and false-alarm rates together. In the grazing model, with dense records spanning the whole change, Kendall’s for variance and autocorrelation separates well (AUC 0.98 and 0.94), but holding false alarms to 5 % needs a cut-off of about 0.8; with sparser records autocorrelation fails first, and with only the last short stretch both are close to a coin toss. Noise-induced flips and flickering can happen far from a threshold; selecting only series that have collapsed makes us “see” warnings in a system whose parameters never changed. In 126 real data sets true positives were rare and false positives commoner than false negatives. The next chapter takes another angle: instead of forecasting when a threshold will arrive, it asks how far away the threshold is and how that distance can be increased.
References
Fan, Y. (2026). Phase Transitions, Critical Slowing Down and Resilience in Complex Dynamical Systems: From Saddle-Node Bifurcations and Early-Warning Signals to Cascading Failures, Read Beside Classical Chinese Thought on Incipient Change [复杂系统相变、临界慢化与生态韧性的非线性动力学] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23770082 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).