What does Context-Parametric Rule Distillation and Online Calibration conclude, in numbers, about “Threshold Drift: Discounted Beta Updates, Sliding Windows and a Cumulative-Sum Alarm”?
Answer: Chapter 5 compares discounting, sliding windows and a cumulative-sum alarm; under a step change the sliding window's total regret is 33.0.
Scope of the numbers: Parameters are illustrative and all calculations are generated by the book's programs. This is general business analysis, not investment, legal or medical advice, and it describes no real customer or sale.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 33.0
| sessions 12,000, change at 5,000, runs 30 | ||||
| policy | regret before | regret after | total | p90 total |
| fixed | 0.0 | 84.0 | 84.0 | 84.0 |
| stationary | 10.1 | 60.7 | 70.8 | 85.4 |
| discount | 21.2 | 29.0 | 50.2 | 57.9 |
| window | 11.2 | 21.8 | 33.0 | 49.4 |
| effective memory of the discount 0.998: 500 sessions |
Show the full program
# Chapter 5. p51: thresholds drift when traffic changes. Two copy tiers;
# at session 5,000 of 12,000 the traffic mix changes and their true pay
# rates swap order: tier 0 goes 0.045 -> 0.030, tier 1 goes
# 0.032 -> 0.042 (illustrative). 30 runs per policy:
# fixed : tier 0 forever (the threshold set before the change)
# stationary: Thompson sampling with all history weighted equally
# discount : Thompson with counts multiplied by 0.998 each session
# window : Thompson on the last 1,500 sessions of each tier only
# Regret is measured against the best tier of each period.
import collections
import math
import random
T, CHANGE, RUNS = 12000, 5000, 30
BEFORE, AFTER = (0.045, 0.032), (0.030, 0.042)
def run(policy, rng):
a = [0.0, 0.0]
b = [0.0, 0.0]
hist = [collections.deque(), collections.deque()]
lost_pre, lost_post = [], []
for t in range(T):
p = BEFORE if t < CHANGE else AFTER
if policy == "fixed":
arm = 0
else:
draws = [rng.betavariate(1.0 + a[i], 1.0 + b[i])
for i in range(2)]
arm = 0 if draws[0] >= draws[1] else 1
r = 1 if rng.random() < p[arm] else 0
if policy == "discount":
for i in range(2):
a[i] *= 0.998
b[i] *= 0.998
a[arm] += r
b[arm] += 1 - r
if policy == "window":
hist[arm].append(r)
if len(hist[arm]) > 1500:
old = hist[arm].popleft()
a[arm] -= old
b[arm] -= 1 - old
gap = max(p) - p[arm]
(lost_pre if t < CHANGE else lost_post).append(gap)
return math.fsum(lost_pre), math.fsum(lost_post)
def main():
seeds = {"fixed": random.Random(5101),
"stationary": random.Random(5102),
"discount": random.Random(5103),
"window": random.Random(5104)}
print("sessions {:,}, change at {:,}, runs {}".format(T, CHANGE, RUNS))
print(" policy regret before regret after total p90 total")
for pol in ("fixed", "stationary", "discount", "window"):
pre, post, tot = [], [], []
for _ in range(RUNS):
x, y = run(pol, seeds[pol])
pre.append(x)
post.append(y)
tot.append(x + y)
tot.sort()
print(" {:<11} {:>10.1f} {:>10.1f} {:>6.1f} {:>6.1f}".format(
pol, math.fsum(pre) / RUNS, math.fsum(post) / RUNS,
math.fsum(tot) / RUNS, tot[26]))
print("effective memory of the discount 0.998: {:.0f} sessions".format(
1.0 / (1.0 - 0.998)))
if __name__ == "__main__":
main()Single source and citation
Fan, Y. (2026). Next-Generation Knowledge Distillation, Context-Parametric Compilation and Live-Conversion Feedback Calibration (Version 1.0.0), Chapter 5: Threshold Drift: Discounted Beta Updates, Sliding Windows and a Cumulative-Sum Alarm. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-377 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026377,
author = {Fan, Yafu},
title = {Next-Generation Knowledge Distillation, Context-Parametric Compilation and Live-Conversion Feedback Calibration},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Simpson's Reversal: Why a Rule Without Its Context Can Point the Wrong Way
- Chapter 2 · Context-Parametric Compilation: Squeezing a Decision Table into Decision Lists and Ordered Decision Diagrams
- Chapter 3 · Resolving Conflicts Across Volumes: Do More Rules Make a Pack More Accurate?
- Chapter 4 · Online Bayesian Calibration: Giving Every Rule a Beta Distribution That Grows
- Chapter 6 · Calibration and the Whole Pipeline: the Brier Partition, Binned Calibration Error and a Net-Benefit Simulation
- Chapter 7 · Classical Sources: Wheelwright Bian and Aristotle on Equity
- Chapter 8 · Closing the Loop: Field Tools and Checker Specifications, the Risk Ledger and a Pre-registered Test