5D Library › 宏观流动性体制转换、极值尾部相依与杠铃资产配置 · English

第一章 隐马尔可夫模型与波动率体制

平静与紧张

把一段很长的日收益率序列画出来,最醒目的往往不是平均水平,而是波动的「成团」:大的涨跌后面跟着大的涨跌,平静的日子后面跟着平静的日子。第 20 本用 GARCH 模型描述过这种波动聚集:今天的方差是昨天的方差与昨天收益平方的加权和,波动连续地变化 (Bollerslev 1986)。本章换一个角度:假设市场在少数几个离散的「体制」之间切换,每个体制有自己的均值与波动率,而我们看不到当下处于哪个体制,只能从收益中推断。这就是隐马尔可夫模型(HMM)。

汉密尔顿的马尔可夫切换模型

证据

汉密尔顿 1989 年提出用一个不可观测的两状态马尔可夫链驱动时间序列的均值,用来刻画经济周期中的扩张与衰退 (Hamilton 1989)。他用的是美国战后实际 GNP 的季度增长率(1951 年至 1984 年),状态切换的是增长率的均值,而不是波动率。估计得到的扩张与衰退的持续概率约为 0.90 与 0.75,对应的期望持续期约为 10 个季度与 4 个季度;模型推断出的衰退时期与美国国家经济研究局(NBER)公布的衰退日期大体吻合。这一框架后来被广泛推广到收益率的均值、波动率与相关性同时切换的模型,例如昂与贝卡尔特对国际股票配置的研究:他们发现高波动的熊市体制中各国股市的相关性更高,但国际分散化仍然有价值,忽略体制的代价主要出现在可以持有无风险资产的情形 (Ang and Bekaert 2002)。

作者的框架用「汉密尔顿 1989」作为波动率体制识别的出处。核对后应当说明:汉密尔顿的原始模型切换的是 GNP 增长的均值;用 HMM 识别波动率体制是同一数学框架的推广,不是原文的内容。

模型与三个算法

设隐藏状态 st∈{1,…,K}s_t \in \{1, \dots, K\} 是一条马尔可夫链,转移概率 pjk=P(st+1=k∣st=j)p_{jk} = P(s_{t+1} = k \mid s_t = j);给定状态,观测到的收益 xt∼N(μst,σst2)x_t \sim N(\mu_{s_t}, \sigma_{s_t}^2)。需要解决三个问题:已知参数时,每一天处于各状态的概率是多少(滤波与平滑);参数怎样估计(鲍姆—韦尔奇算法);最可能的整条状态路径是什么(维特比算法)。

前向—后向。 前向变量 αt(k)=P(st=k∣x1,…,xt)\alpha_t(k) = P(s_t = k \mid x_1, \dots, x_t) 按递推

αt(k)∝bk(xt)∑jαt−1(j)pjk,bk(x)=φ(x−μkσk)/σk\alpha_t(k) \propto b_k(x_t) \sum_j \alpha_{t-1}(j)\,p_{jk}, \qquad b_k(x) = \varphi\!\left(\frac{x – \mu_k}{\sigma_k}\right)\Big/\sigma_k

计算,每一步除以归一化常数 ctc_t,这些常数的对数之和就是对数似然 log⁡L=∑tlog⁡ct\log L = \sum_t \log c_t。后向递推给出 βt(j)=∑kpjkbk(xt+1)βt+1(k)/ct+1\beta_t(j) = \sum_k p_{jk}\, b_k(x_{t+1})\, \beta_{t+1}(k) / c_{t+1},两者相乘得到用全部数据推断的平滑概率 γt(k)=αt(k)βt(k)\gamma_t(k) = \alpha_t(k)\beta_t(k)。αt\alpha_t 只用到当天为止的信息,是实时可得的滤波概率;γt\gamma_t 用到了未来,只有事后才能算出。

鲍姆—韦尔奇(EM)。 鲍姆等人 1970 年给出的算法,后来被登普斯特、莱尔德与鲁宾归入一般的 EM 框架 (Baum et al. 1970; Dempster et al. 1977):在 E 步用前向—后向算出 γt(k)\gamma_t(k) 与相邻两天的联合概率 ξt(j,k)\xi_t(j,k),在 M 步用它们作权重更新参数:

p̂jk=∑tξt(j,k)∑t∑k′ξt(j,k′),μ̂k=∑tγt(k)xt∑tγt(k),σ̂k2=∑tγt(k)(xt−μ̂k)2∑tγt(k).\hat p_{jk} = \frac{\sum_t \xi_t(j,k)}{\sum_t \sum_{k'} \xi_t(j,k')}, \qquad \hat\mu_k = \frac{\sum_t \gamma_t(k)\, x_t}{\sum_t \gamma_t(k)}, \qquad \hat\sigma_k^2 = \frac{\sum_t \gamma_t(k)\,(x_t – \hat\mu_k)^2}{\sum_t \gamma_t(k)}.

每一轮迭代都不会降低似然,但只保证收敛到某个局部极大值。

维特比。 维特比 1967 年为卷积码译码提出的动态规划 (Viterbi 1967):对每个状态记住「到达这里的最可能路径」的对数概率与上一步来自哪里,最后从终点回溯,得到使联合概率最大的整条路径。它回答的问题与平滑概率不同:平滑概率是逐日的边际判断,维特比路径是整体上最可能的一条序列。

下面的程序用一个两体制模型模拟 2,500 个交易日(约十年):平静体制日均收益 0.04%、日波动 0.8%,紧张体制日均收益 −0.08%、日波动 2.0%,平静体制保持的概率 0.99,紧张体制 0.97。然后从三种不同的初值出发拟合。

推导

# Two-state Gaussian HMM for daily returns (%), fitted by Baum-Welch (EM) with scaled
# forward-backward recursions and decoded by Viterbi. SIMULATED, ILLUSTRATIVE: calm
# (mean 0.04, sd 0.8), stress (mean -0.08, sd 2.0); P(stay) 0.99 and 0.97; 2,500 days.
import math, random

TRUE = dict(mu=[0.04, -0.08], sd=[0.8, 2.0], P=[[0.99, 0.01], [0.03, 0.97]], pi=[1, 0])

def simulate(T=2500, seed=1989, par=TRUE):
    rng = random.Random(seed); s = 0 if rng.random() < par["pi"][0] else 1; x, z = [], []
    for _ in range(T):
        z.append(s); x.append(rng.gauss(par["mu"][s], par["sd"][s]))
        s = s if rng.random() < par["P"][s][s] else 1 - s
    return x, z

def dens(x, mu, sd):
    return math.exp(-0.5 * ((x - mu) / sd) ** 2) / (sd * math.sqrt(2 * math.pi))

def forward(x, mu, sd, P, pi):               # a[t][k] = P(state k | x_1..x_t)
    K, a, c = len(mu), [], []
    for t, xt in enumerate(x):
        b = [dens(xt, mu[k], sd[k]) for k in range(K)]
        f = ([pi[k] * b[k] for k in range(K)] if t == 0 else
             [b[k] * sum(a[-1][j] * P[j][k] for j in range(K)) for k in range(K)])
        s = sum(f); c.append(s); a.append([v / s for v in f])
    return a, c                              # log L = sum of log c

def backward(x, mu, sd, P, c):
    K, T = len(mu), len(x); be = [[1.0] * K for _ in range(T)]
    for t in range(T - 2, -1, -1):
        b = [dens(x[t + 1], mu[k], sd[k]) for k in range(K)]
        be[t] = [sum(P[j][k] * b[k] * be[t + 1][k] for k in range(K)) / c[t + 1]
                 for j in range(K)]
    return be

def fit(x, mu, sd, P, pi, tol=1e-7, max_iter=500):
    K, T, old = len(mu), len(x), -1e300
    for it in range(1, max_iter + 1):
        a, c = forward(x, mu, sd, P, pi); be = backward(x, mu, sd, P, c)
        ll = sum(math.log(v) for v in c)
        g = [[a[t][k] * be[t][k] for k in range(K)] for t in range(T)]
        xi = [[0.0] * K for _ in range(K)]
        for t in range(T - 1):
            b = [dens(x[t + 1], mu[k], sd[k]) for k in range(K)]
            for j in range(K):
                for k in range(K):
                    xi[j][k] += a[t][j] * P[j][k] * b[k] * be[t + 1][k] / c[t + 1]
        P = [[xi[j][k] / sum(xi[j]) for k in range(K)] for j in range(K)]; pi = g[0][:]
        for k in range(K):
            w = sum(r[k] for r in g); mu[k] = sum(r[k] * v for r, v in zip(g, x)) / w
            sd[k] = max(1e-3, math.sqrt(sum(r[k] * (v - mu[k]) ** 2
                                            for r, v in zip(g, x)) / w))
        if ll - old < tol:
            break
        old = ll
    return dict(mu=mu, sd=sd, P=P, pi=pi, ll=ll, iters=it, gamma=g)

def viterbi(x, mu, sd, P, pi):
    K = len(mu); lg = lambda v: math.log(max(v, 1e-300)); back = []
    d = [lg(pi[k]) + lg(dens(x[0], mu[k], sd[k])) for k in range(K)]
    for xt in x[1:]:
        best = [max(range(K), key=lambda j: d[j] + lg(P[j][k])) for k in range(K)]
        d = [d[best[k]] + lg(P[best[k]][k]) + lg(dens(xt, mu[k], sd[k]))
             for k in range(K)]
        back.append(best)
    s = [max(range(K), key=lambda k: d[k])]
    for best in reversed(back):
        s.append(best[s[-1]])
    return s[::-1]

if __name__ == "__main__":
    x, z = simulate(); P0 = [[0.95, 0.05], [0.05, 0.95]]
    print(f"simulated {len(x)} days; true share of stress days {sum(z) / len(z):.3f}")
    for name, m0, s0 in (("A (calm first)", [0.0, 0.0], [0.5, 1.5]),
                         ("B (stress first)", [0.0, 0.0], [1.5, 0.5]),
                         ("C (nearly equal)", [0.01, -0.01], [1.0, 1.01])):
        r = fit(x, m0, s0, P0, [0.5, 0.5])
        print(f"start {name}: log L {r['ll']:.1f} after {r['iters']} iterations")
        for k in range(2):
            p = r["P"][k][k]
            print(f"   state {k}: mean {r['mu'][k]:+.3f}  sd {r['sd'][k]:.3f}"
                  f"  P(stay) {p:.4f}  duration {1 / (1 - p):6.1f} days")
    r = fit(x, [0.0, 0.0], [0.5, 1.5], P0, [0.5, 0.5])
    path = viterbi(x, r["mu"], r["sd"], r["P"], r["pi"])
    smooth = [1 if g[1] > 0.5 else 0 for g in r["gamma"]]
    agree = lambda s: sum(p == q for p, q in zip(s, z)) / len(z)
    print(f"Viterbi path agrees with the true states on {agree(path):.1%} of days;"
          f" smoothed P > 0.5 on {agree(smooth):.1%}")
    print("true durations: calm 100 days, stress 33.3 days")
# simulated 2500 days; true share of stress days 0.142
# start A (calm first): log L -3360.1 after 22 iterations
#    state 0: mean +0.030  sd 0.801  P(stay) 0.9933  duration  148.8 days
#    state 1: mean +0.022  sd 1.890  P(stay) 0.9579  duration   23.8 days
# start B (stress first): log L -3360.1 after 22 iterations
#    state 0: mean +0.022  sd 1.890  P(stay) 0.9579  duration   23.8 days
#    state 1: mean +0.030  sd 0.801  P(stay) 0.9933  duration  148.8 days
# start C (nearly equal): log L -3360.1 after 20 iterations
#    state 0: mean +0.030  sd 0.801  P(stay) 0.9933  duration  148.8 days
#    state 1: mean +0.022  sd 1.890  P(stay) 0.9579  duration   23.8 days
# Viterbi path agrees with the true states on 95.4% of days; smoothed P > 0.5 on 96.0%
# true durations: calm 100 days, stress 33.3 days

三个初值都收敛到同一个对数似然 −3360.1,说明这里没有遇到坏的局部极值;但初值 B 得到的「状态 0」是紧张体制、「状态 1」是平静体制,与另外两次正好对调。这就是标签切换:似然对状态的编号是对称的,模型本身不知道哪个状态叫「平静」。实践中要用一条明确的规则(例如按波动率从小到大排序)给状态命名;在贝叶斯混合模型中这个问题更棘手,需要专门的处理 (Stephens 2000)。

更值得注意的是估计误差。真实的持续期是平静 100 天、紧张 33.3 天,估计值却是 148.8 天与 23.8 天;紧张体制的日均收益估计为 +0.022%,连符号都与真实的 −0.08% 相反。波动率估得很准(0.801 与 1.890),因为每天的收益都携带关于方差的信息;而均值与持续期只能从少数几次体制切换中推断。维特比路径与真实状态在 95.4% 的日子一致,平滑概率大于 0.5 的划分一致率为 96.0%:事后看,体制划分相当可靠,但描述体制的参数并不可靠。

持续期与转移概率

在马尔可夫链中,体制每一天都以概率 1−pii1 – p_{ii} 结束,所以持续期 DD 服从几何分布:

P(D=d)=(1−pii)piid−1,E[D]=11−pii,P(D>d)=piid.P(D = d) = (1 – p_{ii})\,p_{ii}^{\,d-1}, \qquad E[D] = \frac{1}{1 – p_{ii}}, \qquad P(D > d) = p_{ii}^{\,d}.

作者框架中的「期望持续期 = 1/(1 − p_ii)」就是这个结果,推导无误。但它有两个常被忽略的含义。第一,几何分布是无记忆的:一个已经持续了 100 天的体制,剩余持续期的分布与刚开始时完全一样。这是模型的假设,不是对现实的发现;如果真实的经济周期或危机有「越久越可能结束」(或「越久越难结束」)的性质,就需要持续期依赖的模型。第二,piip_{ii} 接近 1 时,持续期对它极其敏感:p=0.97p = 0.97 时期望持续期 33 天,p=0.98p = 0.98 时 50 天。下面的程序先验证几何分布的性质,再看在已知真实状态的最好情形下,从有限样本估计持续期有多难。

推导

# Regime durations in a Markov chain: geometric with mean 1/(1 - p_ii), memoryless, and
# hard to estimate from short samples even when the states are KNOWN (the best case).
# ILLUSTRATIVE: daily chain with P(stay calm) 0.99, P(stay stressed) 0.97; quarterly chain
# with the persistence Hamilton (1989) reported for US real GNP, about 0.90 (expansion)
# and 0.75 (recession); 131 quarters as in his post-war sample.
import random, statistics

def chain(T, p_stay, rng, s=0):
    z = []
    for _ in range(T):
        z.append(s)
        s = s if rng.random() < p_stay[s] else 1 - s
    return z

def spells(z):
    out, start = [], 0
    for t in range(1, len(z) + 1):
        if t == len(z) or z[t] != z[start]:
            out.append((z[start], t - start, start > 0 and t < len(z)))
            start = t
    return out                # (state, length, complete = not cut by the sample ends)

def p_hat(z, k):
    n = sum(1 for t in range(len(z) - 1) if z[t] == k)
    stay = sum(1 for t in range(len(z) - 1) if z[t] == k and z[t + 1] == k)
    return stay / n if n else float("nan")

if __name__ == "__main__":
    rng = random.Random(2)
    z = chain(2_000_000, (0.99, 0.97), rng)
    sp = [(s, d) for s, d, ok in spells(z) if ok]
    for k, p in ((0, 0.99), (1, 0.97)):
        d = [n for s, n in sp if s == k]
        long = [n - 100 for n in d if n > 100]
        print(f"state {k}: p = {p}: 1/(1-p) = {1 / (1 - p):5.1f}; simulated mean"
              f" {statistics.mean(d):5.1f};"
              f" P(D > 50) {sum(n > 50 for n in d) / len(d):.3f}"
              f" (p^50 {p ** 50:.3f})")
        print(f"   spells already past day 100: mean further length"
              f" {statistics.mean(long):6.1f}")
    print("10 years of daily data, states KNOWN, 2,000 samples:")
    est, eps = [], []
    for _ in range(2000):
        z = chain(2500, (0.99, 0.97), rng)
        p = p_hat(z, 1)
        est.append(1 / (1 - p) if p < 1 else float("inf"))
        eps.append(sum(1 for t in range(1, len(z)) if z[t] == 1 and z[t - 1] == 0))
    q = statistics.quantiles(est, n=20)
    print(f"   stress episodes per sample: median {statistics.median(eps):.0f}"
          f" (5-95 %: {statistics.quantiles(eps, n=20)[0]:.0f}"
          f"-{statistics.quantiles(eps, n=20)[-1]:.0f})")
    print(f"   estimated stress duration: median {q[9]:.1f} days,"
          f" 5-95 % {q[0]:.1f}-{q[-1]:.1f}"
          f" (true 33.3)")
    print("131 quarters, p(expansion) 0.904, p(recession) 0.755, states KNOWN,"
          " 2,000 samples:")
    est, rec = [], []
    for _ in range(2000):
        z = chain(131, (0.904, 0.755), rng)
        p = p_hat(z, 1)
        if p == p:
            est.append(1 / (1 - p) if p < 1 else 131.0)
        rec.append(sum(1 for t in range(1, len(z)) if z[t] == 1 and z[t - 1] == 0))
    q = statistics.quantiles(est, n=20)
    print(f"   recessions per sample: median {statistics.median(rec):.0f};"
          f" implied recession duration median {q[9]:.1f} quarters,")
    print(f"   5-95 % {q[0]:.1f}-{q[-1]:.1f} (true {1 / 0.245:.1f})")
# state 0: p = 0.99: 1/(1-p) = 100.0; simulated mean  99.8; P(D > 50) 0.609 (p^50 0.605)
#    spells already past day 100: mean further length   99.0
# state 1: p = 0.97: 1/(1-p) =  33.3; simulated mean  33.1; P(D > 50) 0.217 (p^50 0.218)
#    spells already past day 100: mean further length   33.1
# 10 years of daily data, states KNOWN, 2,000 samples:
#    stress episodes per sample: median 19 (5-95 %: 13-25)
#    estimated stress duration: median 32.7 days, 5-95 % 22.6-48.1 (true 33.3)
# 131 quarters, p(expansion) 0.904, p(recession) 0.755, states KNOWN, 2,000 samples:
#    recessions per sample: median 9; implied recession duration median 4.0 quarters,
#    5-95 % 2.5-6.6 (true 4.1)

模拟的平均持续期(99.8 天、33.1 天)与 1/(1−p)1/(1-p) 一致,P(D>50)P(D > 50) 与 p50p^{50} 一致;已经超过 100 天的体制,平均还会再持续 99.0 天与 33.1 天,与刚开始时一样——这就是无记忆。十年日数据中紧张期的个数中位数只有 19 次(90% 区间 13 到 25 次),即使每一天的状态都已知,估计的紧张体制持续期的 90% 区间也从 22.6 天到 48.1 天。用汉密尔顿估计的季度参数模拟 131 个季度,每个样本只有约 9 次衰退,衰退持续期的 90% 区间为 2.5 到 6.6 个季度(真实值 4.1)。真实应用中状态还是未知的,误差只会更大。

事后看得清,实时看不清

推导

# (1) Regimes are clearer after the fact: filtered P(stress | data to day t) against
# smoothed P(stress | all data), with the TRUE parameters of p01 (the best case).
# (2) How many states? Data from TWO regimes whose daily shocks are fat-tailed (Student t,
# 4 d.f., unit variance); Gaussian HMMs with 1, 2, 3 states are fitted on 2,500 days and
# scored by BIC and by log-likelihood on the next 2,500 days. All data SIMULATED.
import math, random

MU, SD, P = [0.04, -0.08], [0.8, 2.0], [[0.99, 0.01], [0.03, 0.97]]

def simulate(T, seed, df=None):
    rng = random.Random(seed); s, x, z = 0, [], []
    for _ in range(T):
        e = rng.gauss(0, 1)
        if df:
            e /= math.sqrt(rng.gammavariate(df / 2, 2) / df) * math.sqrt(df / (df - 2))
        z.append(s); x.append(MU[s] + SD[s] * e)
        s = s if rng.random() < P[s][s] else 1 - s
    return x, z

def dens(x, m, s):
    return math.exp(-0.5 * ((x - m) / s) ** 2) / (s * 2.5066282746310002)

def fwd_bwd(x, mu, sd, P, pi):
    K, T = len(mu), len(x); a, c, B = [], [], []
    for t in range(T):
        b = [dens(x[t], mu[k], sd[k]) for k in range(K)]; B.append(b)
        f = ([pi[k] * b[k] for k in range(K)] if t == 0 else
             [b[k] * sum(a[-1][j] * P[j][k] for j in range(K)) for k in range(K)])
        s = sum(f); c.append(s); a.append([v / s for v in f])
    be = [[1.0] * K for _ in range(T)]
    for t in range(T - 2, -1, -1):
        be[t] = [sum(P[j][k] * B[t + 1][k] * be[t + 1][k] for k in range(K)) / c[t + 1]
                 for j in range(K)]
    return a, be, c, B

def fit(x, mu, sd, max_iter=300, tol=1e-5):
    K, T = len(mu), len(x); P = [[0.95 if j == k else 0.05 / max(K - 1, 1)
                                  for k in range(K)] for j in range(K)]
    pi, old = [1 / K] * K, -1e300
    for _ in range(max_iter):
        a, be, c, B = fwd_bwd(x, mu, sd, P, pi); ll = sum(math.log(v) for v in c)
        g = [[a[t][k] * be[t][k] for k in range(K)] for t in range(T)]
        xi = [[sum(a[t][j] * P[j][k] * B[t + 1][k] * be[t + 1][k] / c[t + 1]
                   for t in range(T - 1)) for k in range(K)] for j in range(K)]
        P = [[v / sum(row) for v in row] for row in xi]; pi = g[0]
        for k in range(K):
            w = sum(r[k] for r in g)
            mu[k] = sum(r[k] * v for r, v in zip(g, x)) / w
            v2 = sum(r[k] * (v - mu[k]) ** 2 for r, v in zip(g, x)) / w
            sd[k] = max(0.05, math.sqrt(v2))
        if ll - old < tol:
            break
        old = ll
    return mu, sd, P, pi, ll

def loglik(x, mu, sd, P, pi):
    return sum(math.log(v) for v in fwd_bwd(x, mu, sd, P, pi)[2])

def lags(z, prob, thr=0.5):
    out, t = [], 1
    while t < len(z):
        if z[t] == 1 and z[t - 1] == 0:
            e = t
            while e < len(z) and z[e] == 1:
                e += 1
            if e - t >= 10:
                hit = next((u - t for u in range(t, e) if prob[u] > thr), None)
                out.append(hit)
            t = e
        t += 1
    return out

if __name__ == "__main__":
    x, z = simulate(20000, 7)
    a, be, c, _ = fwd_bwd(x, MU, SD, P, [1.0, 0.0])
    filt = [r[1] for r in a]; smo = [r[1] * q[1] for r, q in zip(a, be)]
    for name, pr in (("filtered", filt), ("smoothed", smo)):
        L = lags(z, pr); found = sorted(v for v in L if v is not None)
        fa = sum(1 for p, s in zip(pr, z) if p > 0.5 and s == 0) / z.count(0)
        print(f"{name:8s}: {len(L)} stress spells >= 10 days;"
              f" detected {len(found) / len(L):.0%};"
              f" median lag {found[len(found) // 2]} days,")
        print(f"{'':10s}90th percentile {found[int(0.9 * len(found))]} days;"
              f" P > 0.5 on {fa:.1%} of calm days (false alarms)")
    x, _ = simulate(5000, 11, df=4); ins, oos = x[:2500], x[2500:]
    print("data: 2 regimes with Student-t(4) shocks; Gaussian HMMs fitted on days 1-2500")
    print(" K   params   log L in-sample     BIC    log L days 2501-5000 (per day)   sd")
    starts = {1: ([0.0], [1.0]), 2: ([0.05, -0.05], [0.7, 1.8]),
              3: ([0.05, 0.0, -0.1], [0.6, 1.0, 2.2])}
    for K, (m0, s0) in starts.items():
        mu, sd, Pk, pi, ll = fit(ins, m0[:], s0[:])
        npar = 2 * K + K * (K - 1) + (K - 1)
        bic = -2 * ll + npar * math.log(len(ins)); lo = loglik(oos, mu, sd, Pk, pi)
        sds = ", ".join(f"{v:.2f}" for v in sd)
        print(f" {K}   {npar:5d}   {ll:10.1f}    {bic:9.1f}    {lo:10.1f}"
              f" ({lo / len(oos):+.4f})"
              f"   {sds}")
# filtered: 112 stress spells >= 10 days; detected 99%; median lag 2 days,
#           90th percentile 7 days; P > 0.5 on 3.5% of calm days (false alarms)
# smoothed: 112 stress spells >= 10 days; detected 99%; median lag 0 days,
#           90th percentile 3 days; P > 0.5 on 1.7% of calm days (false alarms)
# data: 2 regimes with Student-t(4) shocks; Gaussian HMMs fitted on days 1-2500
#  K   params   log L in-sample     BIC    log L days 2501-5000 (per day)   sd
#  1       2      -3688.8       7393.3       -3835.9 (-1.5343)   1.06
#  2       7      -3300.3       6655.3       -3333.5 (-1.3334)   0.65, 1.98
#  3      14      -3262.7       6634.9       -3310.6 (-1.3243)   0.53, 1.18, 1.96

程序的第一部分用真实参数(最好情形,现实中参数也要估计)计算 20,000 天模拟数据的滤波概率与平滑概率。对 112 次持续至少 10 天的紧张期,滤波概率超过 0.5 的中位滞后为 2 天,第 90 百分位为 7 天;平滑概率的对应数字为 0 天与 3 天。平静日中滤波概率误报(超过 0.5)的比例为 3.5%,平滑概率为 1.7%。图 1 显示了前 1,000 天:平滑概率(虚线)在紧张期两端干净地上升和下降,滤波概率(实线)则在体制开始后才追上,并在平静期出现零星的尖峰。

模拟的日收益率(上)与紧张体制的概率(下),灰色为真实的紧张体制。实线为只用当天以前数据的滤波概率,虚线为用全部数据的平滑概率(真实参数,示意)。

这就是「体制往往只在事后才看得清」的数学含义:平滑概率之所以干净,是因为它借用了未来的数据。任何用 HMM 做实时决策的规则,只能使用滤波概率;用平滑概率做的回测会系统地高估规则的效果,因为它在每个时点都偷看了未来。

状态数怎么选? 程序的第二部分让数据来自两个体制,但每个体制内的冲击是自由度为 4 的学生 t 分布(厚尾),然后拟合 1、2、3 个状态的高斯 HMM。贝叶斯信息准则(BIC,越小越好)从 7393.3 降到 6655.3 再降到 6634.9,样本外的对数似然也是三状态最好(每天 −1.3243 对 −1.3334)。也就是说,信息准则与样本外检验都「选出」了第三个体制,而它在数据生成过程中并不存在:三状态模型用一个波动率 1.18 的中间状态去拟合每个体制内的厚尾。所以状态数不是从数据中「发现」的,而是模型设定(体制内分布形状)与数据共同决定的;把三个状态解释为三种经济含义不同的市场环境,需要模型之外的理由。

注

局限汇总。 (1)状态数与体制内分布是建模选择,换一个选择,体制的个数与含义都会变。(2)标签需要用明确规则命名。(3)鲍姆—韦尔奇只保证局部极大,应从多个初值出发。(4)样本内拟合好不等于样本外预测好;持续期、均值等参数的不确定性往往很大。(5)实时只能用滤波概率,它比平滑概率滞后并有误报。(6)马尔可夫假设意味着持续期无记忆,这是假设而不是事实。

假说 · 尚未验证

作者的假说(尚未确立): 以 HMM 滤波概率触发的再平衡规则(例如紧张概率超过 0.5 时把风险资产比例降低一半),在样本外的风险调整后收益高于固定比例再平衡。什么样的结果能证明它是错的: 在事先锁定的样本外窗口与交易成本假设下,两种规则的夏普比率或最大回撤之差的置信区间包含零或偏向固定比例。第八章说明,这类检验需要的数据年数极长,所以本书选择了另一项更有把握度的检验。

本章小结

隐马尔可夫模型把市场描述为在几个看不见的体制之间切换。前向—后向算法给出滤波与平滑概率,鲍姆—韦尔奇算法估计参数,维特比算法给出最可能的状态路径。汉密尔顿 1989 年的原始模型切换的是 GNP 增长的均值,波动率体制是其推广。期望持续期 1/(1−pii)1/(1-p_{ii}) 在无记忆假设下成立,但从十年数据估计的持续期误差很大。事后的平滑概率能把体制划分得相当清楚,实时的滤波概率则有滞后与误报;收益厚尾时,信息准则会多选出一个不存在的体制。体制识别因此是一种依赖假设的概率划分,而不是对市场状态的直接观测。

参考文献

Ang, Andrew, and Geert Bekaert. 2002. “International Asset Allocation with Regime Shifts.” The Review of Financial Studies 15 (4): 1137–87. https://doi.org/10.1093/rfs/15.4.1137.
Baum, Leonard E., Ted Petrie, George Soules, and Norman Weiss. 1970. “A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains.” The Annals of Mathematical Statistics 41 (1): 164–71. https://doi.org/10.1214/aoms/1177697196.
Bollerslev, Tim. 1986. “Generalized Autoregressive Conditional Heteroskedasticity.” Journal of Econometrics 31 (3): 307–27. https://doi.org/10.1016/0304-4076(86)90063-1.
Dempster, Arthur P., Nan M. Laird, and Donald B. Rubin. 1977. “Maximum Likelihood from Incomplete Data via the EM Algorithm.” Journal of the Royal Statistical Society: Series B (Methodological) 39 (1): 1–38. https://doi.org/10.1111/j.2517-6161.1977.tb01600.x.
Hamilton, James D. 1989. “A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle.” Econometrica 57 (2): 357–84. https://doi.org/10.2307/1912559.
Stephens, Matthew. 2000. “Dealing with Label Switching in Mixture Models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 62 (4): 795–809. https://doi.org/10.1111/1467-9868.00265.
Viterbi, Andrew J. 1967. “Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm.” IEEE Transactions on Information Theory 13 (2): 260–69. https://doi.org/10.1109/TIT.1967.1054010.

Fan, Y. (2026). Macro-Liquidity Regime Switching, Extreme Tail Dependence and Barbell Asset Allocation: Vine Copulas, Hidden Markov Models and Sovereign Buffer Dynamics [宏观流动性体制转换、极值尾部相依与杠铃资产配置] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23828049 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).