What does Context-Parametric Rule Distillation and Online Calibration conclude, in numbers, about “Online Bayesian Calibration: Giving Every Rule a Beta Distribution That Grows”?
Answer: Chapter 4 calibrates rules online with Beta posteriors and Thompson sampling, whose mean regret over forty thousand sessions is 50.2; detecting a rise in pay rate from 0.040 to 0.044 needs 39,475 visitors per arm.
Scope of the numbers: Parameters are illustrative and all calculations are generated by the book's programs. This is general business analysis, not investment, legal or medical advice, and it describes no real customer or sale.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.040, 0.044, 39475
| A. minimum exposures for a 90% interval no wider than w | |
| w | 0.020: n >= 992 |
| w | 0.010: n >= 4,100 |
| w | 0.005: n >= 16,571 |
| B. sessions per arm to detect 0.040 -> p1 (alpha 0.05, 0.8) | |
| p1 | 0.044 (lift 10%): 39,475 |
| p1 | 0.048 (lift 20%): 10,317 |
| p1 | 0.052 (lift 30%): 4,783 |
| p1 | 0.060 (lift 50%): 1,863 |
| C. true rate stays 0.040; chance a window looks < | 0.032 |
| window | 100 sessions: 0.416 |
| window | 300 sessions: 0.244 |
| window 1,000 sessions | 0.113 |
| window 3,000 sessions | 0.014 |
Show the full program
# Chapter 4. p43: how many sessions before a rule's threshold may move?
# Part A: smallest exposure count n at which the 90% posterior interval
# of a rule with prior Beta(2, 48) and observed rate 0.04 is no wider
# than a preset ceiling w (k = round(0.04 n)).
# Part B: sessions per arm for a two-sided test (alpha 0.05, power 0.8)
# that a re-tuned rule moves the pay rate from 0.040 to p1.
# Part C: fixed-threshold edits made on small samples: how often does a
# rule whose true rate is unchanged look 20% worse by chance?
import math
import random
import statistics
def betacf(a, b, x):
tiny = 1e-300
qab, qap, qam = a + b, a + 1.0, a - 1.0
c, d = 1.0, 1.0 - qab * x / qap
d = 1.0 / (d if abs(d) > tiny else tiny)
h = d
for m in range(1, 400):
m2 = 2 * m
aa = m * (b - m) * x / ((qam + m2) * (a + m2))
d = 1.0 + aa * d
d = 1.0 / (d if abs(d) > tiny else tiny)
c = 1.0 + aa / c
c = c if abs(c) > tiny else tiny
h *= d * c
aa = -(a + m) * (qab + m) * x / ((a + m2) * (qap + m2))
d = 1.0 + aa * d
d = 1.0 / (d if abs(d) > tiny else tiny)
c = 1.0 + aa / c
c = c if abs(c) > tiny else tiny
delta = d * c
h *= delta
if abs(delta - 1.0) < 1e-14:
break
return h
def ibeta(a, b, x):
if x <= 0.0:
return 0.0
if x >= 1.0:
return 1.0
lb = (math.lgamma(a + b) - math.lgamma(a) - math.lgamma(b)
+ a * math.log(x) + b * math.log(1.0 - x))
if x < (a + 1.0) / (a + b + 2.0):
return math.exp(lb) * betacf(a, b, x) / a
return 1.0 - math.exp(lb) * betacf(b, a, 1.0 - x) / b
def bquant(a, b, q):
lo, hi = 0.0, 1.0
for _ in range(80):
mid = 0.5 * (lo + hi)
if ibeta(a, b, mid) < q:
lo = mid
else:
hi = mid
return 0.5 * (lo + hi)
def width(n, p=0.04):
k = round(p * n)
a, b = 2.0 + k, 48.0 + n - k
return bquant(a, b, 0.95) - bquant(a, b, 0.05)
def min_n(w):
lo, hi = 1, 200000
while lo < hi:
mid = (lo + hi) // 2
if width(mid) <= w:
hi = mid
else:
lo = mid + 1
return lo
def per_arm(p0, p1, alpha=0.05, power=0.8):
z = statistics.NormalDist()
za, zb = z.inv_cdf(1 - alpha / 2), z.inv_cdf(power)
pb = (p0 + p1) / 2
num = (za * math.sqrt(2 * pb * (1 - pb))
+ zb * math.sqrt(p0 * (1 - p0) + p1 * (1 - p1)))
return math.ceil((num / (p1 - p0)) ** 2)
def main():
print("A. minimum exposures for a 90% interval no wider than w")
for w in (0.02, 0.01, 0.005):
print(" w = {:.3f}: n >= {:,}".format(w, min_n(w)))
print("B. sessions per arm to detect 0.040 -> p1 (alpha 0.05, 0.8)")
for p1 in (0.044, 0.048, 0.052, 0.060):
print(" p1 = {:.3f} (lift {:.0%}): {:,}".format(
p1, p1 / 0.04 - 1, per_arm(0.040, p1)))
print("C. true rate stays 0.040; chance a window looks <= 0.032")
rng = random.Random(4301)
for n in (100, 300, 1000, 3000):
hit = 0
for _ in range(4000):
k = 0
for _ in range(n):
k += 1 if rng.random() < 0.04 else 0
hit += 1 if k / n <= 0.032 else 0
print(" window {:>5,} sessions: {:.3f}".format(n, hit / 4000))
if __name__ == "__main__":
main()Single source and citation
Fan, Y. (2026). Next-Generation Knowledge Distillation, Context-Parametric Compilation and Live-Conversion Feedback Calibration (Version 1.0.0), Chapter 4: Online Bayesian Calibration: Giving Every Rule a Beta Distribution That Grows. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-377 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026377,
author = {Fan, Yafu},
title = {Next-Generation Knowledge Distillation, Context-Parametric Compilation and Live-Conversion Feedback Calibration},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Simpson's Reversal: Why a Rule Without Its Context Can Point the Wrong Way
- Chapter 2 · Context-Parametric Compilation: Squeezing a Decision Table into Decision Lists and Ordered Decision Diagrams
- Chapter 3 · Resolving Conflicts Across Volumes: Do More Rules Make a Pack More Accurate?
- Chapter 5 · Threshold Drift: Discounted Beta Updates, Sliding Windows and a Cumulative-Sum Alarm
- Chapter 6 · Calibration and the Whole Pipeline: the Brier Partition, Binned Calibration Error and a Net-Benefit Simulation
- Chapter 7 · Classical Sources: Wheelwright Bian and Aristotle on Equity
- Chapter 8 · Closing the Loop: Field Tools and Checker Specifications, the Risk Ledger and a Pre-registered Test