5D Library › 预测市场微观结构、布雷格曼信息投影与跨盘口无套利边界 · English
第六章 从价格读概率:校准检验、相关结算与修正的代价
偏差存在,然后呢
前五章说明了价格为什么会偏离概率:信念分散、风险态度、资金锁仓、无法执行的套利。一个自然的推论是:如果偏差有规律,就把它修正掉——比如把 0.05 的冷门读成 0.033。本章问三个问题:要多少已结算的合约才能确认偏差?这些合约真的是独立的样本吗?修正之后,预测会不会反而变差?
看见偏差需要多少数据
推导
一参数校准模型。 设真实对数几率是价格对数几率的 倍:。 表示完美校准, 表示冷门被高估、热门被低估。用已结算合约的结果(0 或 1)以最大似然估计 ,并检验 。
推导
# Chapter 6. p12: How many resolved contracts does it take to see the bias?
# Prices are uniform on the log-odds scale in [-3, 3]. The truth is
# logit(p) = beta * logit(price), beta = 1.15
# (beta > 1: longshots overpriced, favourites underpriced; at a price of
# 0.05 the true probability is 0.033). For each sample size, 200 samples
# are drawn; beta is fitted by maximum likelihood (Newton) and the test
# of beta = 1 at the 5 % level is counted. A calibrated market (beta = 1)
# is also run to check the false-positive rate.
import math
import random
def sig(x):
return 1 / (1 + math.exp(-x))
def fit(xs, ys):
b = 1.0
for _ in range(25):
g = h = 0.0
for x, y in zip(xs, ys):
p = sig(b * x)
g += (y - p) * x
h += p * (1 - p) * x * x
b += g / h
if abs(g / h) < 1e-10:
break
return b, 1 / math.sqrt(h)
def run(rng, n, beta, reps=200):
hits, ests = 0, []
for _ in range(reps):
xs = [rng.uniform(-3, 3) for _ in range(n)]
ys = [1 if rng.random() < sig(beta * x) else 0 for x in xs]
b, se = fit(xs, ys)
ests.append(b)
hits += abs(b - 1) / se > 1.96
m = sum(ests) / reps
sd = math.sqrt(sum((e - m) ** 2 for e in ests) / (reps - 1))
return hits / reps, m, sd
if __name__ == "__main__":
rng = random.Random(59012)
print("price 0.05 -> true p {:.3f} when beta = 1.15".format(
sig(1.15 * math.log(0.05 / 0.95))))
print("\n n power (beta=1.15) mean est sd "
"false pos (beta=1)")
for n in (100, 300, 1000, 3000):
pw, m, sd = run(rng, n, 1.15)
fp, _, _ = run(rng, n, 1.0)
print(" {:5d} {:5.3f} {:5.3f} {:5.3f}"
" {:5.3f}".format(n, pw, m, sd, fp))
print("\nReading: a bias that turns a 5 % price into a 3.3 % chance is")
print("invisible in 100 resolved contracts, a coin flip at 1,000 and")
print("reliably seen only at 3,000 - and that assumes the contracts")
print("resolve independently, which the next program questions.")
# price 0.05 -> true p 0.033 when beta = 1.15
#
# n power (beta=1.15) mean est sd false pos (beta=1)
# 100 0.065 1.196 0.253 0.055
# 300 0.255 1.176 0.129 0.055
# 1000 0.575 1.151 0.074 0.035
# 3000 0.985 1.153 0.036 0.045
#
# Reading: a bias that turns a 5 % price into a 3.3 % chance is
# invisible in 100 resolved contracts, a coin flip at 1,000 and
# reliably seen only at 3,000 - and that assumes the contracts
# resolve independently, which the next program questions.时,价格 0.05 的合约真实概率是 0.033。这是一个实质性的偏差:押冷门的人每一元平均只拿回三分之二。但在 100 份已结算合约里,检验只有 0.065 的把握度,几乎与假阳性率相同;300 份时 0.255;1,000 份时 0.575,相当于掷硬币;要到 3,000 份才达到 0.985。假阳性率在各个样本量下都接近 5%,说明检验本身是对的——是数据不够。
合约不是独立的
上面的计算假设每份合约独立结算。真实的预测市场不是这样:一场选举同时挂出几十份合约——各州、得票率区间、投票率——它们由同一批选票决定。第十三段程序构造一个定价完全正确的市场:每份合约的价格就是它的真实概率。合约来自 20 个事件、每个事件 50 份,同一事件内的结果共享一个冲击(高斯 copula,相关系数 ),每份合约的边际概率不变。
推导
# Chapter 6. p13: Correlated resolutions shrink the effective sample.
# 1,000 contracts come from 20 events (50 per event: states, margins,
# turnout lines of one election, say). Prices are perfectly calibrated:
# each contract's price equals its true probability, drawn uniform in
# [0.80, 0.97]. Outcomes share an event-level shock through a Gaussian
# copula with correlation rho, which leaves every marginal probability
# unchanged. The calibration estimate is mean(outcome - price); the
# naive standard error treats contracts as independent. 2,000 runs.
import math
import random
from statistics import NormalDist
E, M, RUNS = 20, 50, 2000
PHI = NormalDist()
def one(rng, rho):
total = var = 0.0
for _ in range(E):
z = rng.gauss(0, 1)
for _ in range(M):
p = rng.uniform(0.80, 0.97)
u = PHI.cdf(math.sqrt(rho) * z + math.sqrt(1 - rho)
* rng.gauss(0, 1))
total += (1 if u < p else 0) - p
var += p * (1 - p)
n = E * M
return total / n, math.sqrt(var) / n
if __name__ == "__main__":
rng = random.Random(59013)
print(" rho true sd naive se ratio effective n"
" naive false positives")
for rho in (0.0, 0.1, 0.3, 0.6):
est, ses = [], []
for _ in range(RUNS):
e, s = one(rng, rho)
est.append(e)
ses.append(s)
sd = math.sqrt(sum(x * x for x in est) / RUNS)
se = sum(ses) / RUNS
fp = sum(abs(e) / s > 1.96 for e, s in zip(est, ses)) / RUNS
print(" {:3.1f} {:7.4f} {:7.4f} {:5.2f} {:6.0f}"
" {:6.3f}".format(rho, sd, se, sd / se,
E * M * (se / sd) ** 2, fp))
print("\nReading: the prices are right by construction, yet with an")
print("event-level correlation of 0.3 a naive test calls them biased")
print("in about half of all samples. 1,000 contracts from 20 events")
print("carry the information of far fewer; inference must cluster by")
print("event.")
# rho true sd naive se ratio effective n naive false positives
# 0.0 0.0098 0.0100 0.98 1036 0.044
# 0.1 0.0170 0.0100 1.70 344 0.243
# 0.3 0.0267 0.0100 2.68 139 0.470
# 0.6 0.0401 0.0100 4.02 62 0.628
#
# Reading: the prices are right by construction, yet with an
# event-level correlation of 0.3 a naive test calls them biased
# in about half of all samples. 1,000 contracts from 20 events
# carry the information of far fewer; inference must cluster by
# event.没有相关时,朴素标准误是对的,假阳性率 0.044。事件内潜变量相关系数(copula )为 0.3 时,估计的真实标准差是朴素标准误的 2.68 倍,1,000 份合约只相当于 139 份独立合约,朴素检验在 0.470 的样本中宣布「有偏差」——尽管价格完全正确。相关系数 0.6 时,有效样本只剩 62 份,假阳性率 0.628。
所以检验预测市场的校准,必须按事件聚类计算标准误 (Cameron and Miller 2015),样本量要按事件数而不是合约数来规划。用「几千份合约」证明某个平台系统性有偏或无偏的研究,若没有聚类,其结论可能只来自几十个事件。
修正偏差的代价
假设偏差确实存在,修正它值得吗?第十四段程序先用布赖尔分数 (Brier 1950) 的 Murphy 分解 (Murphy 1973) 看偏差有多大,再比较两种修正在样本外的表现。
推导
# Chapter 6. p14: Correcting a biased price: when it helps and when it hurts.
# Prices are uniform on log-odds in [-3, 3]; truth logit(p) = 1.15 x
# logit(price), as in p12. Part A splits the market's Brier score into
# reliability (miscalibration, smaller is better), resolution (larger is
# better) and uncertainty (Murphy 1973), 10 price bins, 200,000 contracts.
# Part B learns a correction from n_train resolved contracts and scores
# it on 50,000 fresh ones, 100 repetitions:
# slope refit beta in logit(p) = beta logit(price) (1 parameter)
# bins replace each price by its bin's training frequency (10 bins)
# Reported: mean change in out-of-sample Brier score (negative = better).
import math
import random
def sig(x):
return 1 / (1 + math.exp(-x))
def draw(rng, n):
xs = [rng.uniform(-3, 3) for _ in range(n)]
return [(sig(x), 1 if rng.random() < sig(1.15 * x) else 0) for x in xs]
def bin_of(p):
return min(int(p * 10), 9)
def murphy(rows):
n = len(rows)
base = sum(y for _, y in rows) / n
rel = res = 0.0
for k in range(10):
g = [(p, y) for p, y in rows if bin_of(p) == k]
if not g:
continue
fp = sum(p for p, _ in g) / len(g)
fy = sum(y for _, y in g) / len(g)
rel += len(g) * (fp - fy) ** 2 / n
res += len(g) * (fy - base) ** 2 / n
brier = sum((p - y) ** 2 for p, y in rows) / n
return brier, rel, res, base * (1 - base)
def fit_slope(rows):
b = 1.0
for _ in range(25):
g = h = 0.0
for p, y in rows:
x = math.log(p / (1 - p))
q = sig(b * x)
g += (y - q) * x
h += q * (1 - q) * x * x
b += g / h
return b
def brier(rows, f):
return sum((f(p) - y) ** 2 for p, y in rows) / len(rows)
if __name__ == "__main__":
rng = random.Random(59014)
b, rel, res, unc = murphy(draw(rng, 200000))
print("A. Brier {:.5f}; reliability {:.5f}; resolution {:.5f}"
.format(b, rel, res))
print(" uncertainty {:.5f}; rel - res + unc = {:.5f}"
" (rest: within-bin spread)".format(unc, rel - res + unc))
test = draw(rng, 50000)
raw = brier(test, lambda p: p)
print("\nB. raw out-of-sample Brier {:.5f}".format(raw))
print(" n_train slope fix bins fix (change in Brier)")
for n in (30, 100, 300, 1000, 10000):
ds = db = 0.0
for _ in range(100):
tr = draw(rng, n)
beta = fit_slope(tr)
freq = {}
for k in range(10):
g = [y for p, y in tr if bin_of(p) == k]
freq[k] = sum(g) / len(g) if g else (k + 0.5) / 10
ds += brier(test, lambda p: sig(beta * math.log(p / (1 - p))))
db += brier(test, lambda p: freq[bin_of(p)])
print(" {:7d} {:+.5f} {:+.5f}".format(
n, ds / 100 - raw, db / 100 - raw))
print("\nReading: the bias is real but small - reliability is a tiny")
print("part of the Brier score. A one-parameter correction pays off")
print("once a few hundred contracts are resolved and hurts below that.")
print("A ten-bin correction is worse at every size tried: it throws")
print("away the resolution inside each bin to fix a small bias.")
# A. Brier 0.13714; reliability 0.00053; resolution 0.11262
# uncertainty 0.25000; rel - res + unc = 0.13791 (rest: within-bin spread)
#
# B. raw out-of-sample Brier 0.13675
# n_train slope fix bins fix (change in Brier)
# 30 +0.00289 +0.05731
# 100 +0.00038 +0.01960
# 300 -0.00021 +0.00555
# 1000 -0.00052 +0.00177
# 10000 -0.00057 +0.00035
#
# Reading: the bias is real but small - reliability is a tiny
# part of the Brier score. A one-parameter correction pays off
# once a few hundred contracts are resolved and hurts below that.
# A ten-bin correction is worse at every size tried: it throws
# away the resolution inside each bin to fix a small bias.A 段把 200,000 份合约的布赖尔分数 0.13714 分解为可靠性 0.00053(校准误差)、分辨力 0.11262 与不确定性 0.25000。 的偏差是真实的,但它在总分里只占很小一部分:价格的大部分价值在于区分了可能与不可能的事件,而这一点并不受偏差影响。
B 段用 份训练合约学习修正,在 50,000 份新合约上评分。一参数的斜率修正在 30 份与 100 份时使布赖尔分数变差(+0.00289 与 +0.00038),300 份起才开始改善,10,000 份时改善 −0.00057——而这已接近偏差本身的上限。十格分箱修正在每一种样本量下都更差:30 份时恶化 +0.05731,即使 10,000 份也恶化 +0.00035,因为它用每一格的平均频率替换价格,丢掉了格内的分辨力,只为修正一个很小的偏差。
注
结论的范围。 这些数字来自一个已知只有斜率偏差的模拟市场。真实市场的偏差可能随期限(第三章)、平台与事件类型而变,此时需要更多参数,也就需要更多数据。本章的结论是方向性的:修正偏差的模型越灵活,需要的已结算合约越多,在数据不足时越可能有害。
假说 · 尚未验证
作者的假说(尚未验证): 公开发表的、声称某预测市场「校准良好」或「系统有偏」的研究中,按事件聚类重新计算标准误后,相当一部分结论不再显著。什么结果会证明它错:重新分析后,绝大多数结论的显著性不变。
小结
价格的偏差是真实的,但它小到需要大量数据才能确认:一个把 5% 的价格变成 3.3% 概率的偏差,在 1,000 份独立合约里只有约一半的把握度被发现。而合约并不独立:事件内相关系数 0.3 时,1,000 份合约只相当于 139 份,朴素检验在近一半的样本中误报偏差。偏差在布赖尔分数中只占很小一部分;一参数修正要几百份合约才开始有益,灵活的分箱修正在例子中的每个样本量下都有害。
参考文献
Fan, Y. (2026). Prediction Market Microstructure, Bregman Information Projection and Cross-Book Arbitrage Bounds: Logarithmic Market Scoring Rules, Favorite-Longshot Calibration Bias, Lock-Up Opportunity Cost and Frank-Wolfe Combinatorial Pricing [预测市场微观结构、布雷格曼信息投影与跨盘口无套利边界] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090166 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).