What does Context-Parametric Rule Distillation and Online Calibration conclude, in numbers, about “Online Bayesian Calibration: Giving Every Rule a Beta Distribution That Grows”?

Answer: Chapter 4 calibrates rules online with Beta posteriors and Thompson sampling, whose mean regret over forty thousand sessions is 50.2; detecting a rise in pay rate from 0.040 to 0.044 needs 39,475 visitors per arm.

Scope of the numbers: Parameters are illustrative and all calculations are generated by the book's programs. This is general business analysis, not investment, legal or medical advice, and it describes no real customer or sale.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.040, 0.044, 39475

A. minimum exposures for a 90% interval no wider than w
w0.020: n >= 992
w0.010: n >= 4,100
w0.005: n >= 16,571
B. sessions per arm to detect 0.040 -> p1 (alpha 0.05, 0.8)
p10.044 (lift 10%): 39,475
p10.048 (lift 20%): 10,317
p10.052 (lift 30%): 4,783
p10.060 (lift 50%): 1,863
C. true rate stays 0.040; chance a window looks <0.032
window100 sessions: 0.416
window300 sessions: 0.244
window 1,000 sessions0.113
window 3,000 sessions0.014
Show the full program
# Chapter 4. p43: how many sessions before a rule's threshold may move?
# Part A: smallest exposure count n at which the 90% posterior interval
# of a rule with prior Beta(2, 48) and observed rate 0.04 is no wider
# than a preset ceiling w (k = round(0.04 n)).
# Part B: sessions per arm for a two-sided test (alpha 0.05, power 0.8)
# that a re-tuned rule moves the pay rate from 0.040 to p1.
# Part C: fixed-threshold edits made on small samples: how often does a
# rule whose true rate is unchanged look 20% worse by chance?
import math
import random
import statistics


def betacf(a, b, x):
    tiny = 1e-300
    qab, qap, qam = a + b, a + 1.0, a - 1.0
    c, d = 1.0, 1.0 - qab * x / qap
    d = 1.0 / (d if abs(d) > tiny else tiny)
    h = d
    for m in range(1, 400):
        m2 = 2 * m
        aa = m * (b - m) * x / ((qam + m2) * (a + m2))
        d = 1.0 + aa * d
        d = 1.0 / (d if abs(d) > tiny else tiny)
        c = 1.0 + aa / c
        c = c if abs(c) > tiny else tiny
        h *= d * c
        aa = -(a + m) * (qab + m) * x / ((a + m2) * (qap + m2))
        d = 1.0 + aa * d
        d = 1.0 / (d if abs(d) > tiny else tiny)
        c = 1.0 + aa / c
        c = c if abs(c) > tiny else tiny
        delta = d * c
        h *= delta
        if abs(delta - 1.0) < 1e-14:
            break
    return h


def ibeta(a, b, x):
    if x <= 0.0:
        return 0.0
    if x >= 1.0:
        return 1.0
    lb = (math.lgamma(a + b) - math.lgamma(a) - math.lgamma(b)
          + a * math.log(x) + b * math.log(1.0 - x))
    if x < (a + 1.0) / (a + b + 2.0):
        return math.exp(lb) * betacf(a, b, x) / a
    return 1.0 - math.exp(lb) * betacf(b, a, 1.0 - x) / b


def bquant(a, b, q):
    lo, hi = 0.0, 1.0
    for _ in range(80):
        mid = 0.5 * (lo + hi)
        if ibeta(a, b, mid) < q:
            lo = mid
        else:
            hi = mid
    return 0.5 * (lo + hi)


def width(n, p=0.04):
    k = round(p * n)
    a, b = 2.0 + k, 48.0 + n - k
    return bquant(a, b, 0.95) - bquant(a, b, 0.05)


def min_n(w):
    lo, hi = 1, 200000
    while lo < hi:
        mid = (lo + hi) // 2
        if width(mid) <= w:
            hi = mid
        else:
            lo = mid + 1
    return lo


def per_arm(p0, p1, alpha=0.05, power=0.8):
    z = statistics.NormalDist()
    za, zb = z.inv_cdf(1 - alpha / 2), z.inv_cdf(power)
    pb = (p0 + p1) / 2
    num = (za * math.sqrt(2 * pb * (1 - pb))
           + zb * math.sqrt(p0 * (1 - p0) + p1 * (1 - p1)))
    return math.ceil((num / (p1 - p0)) ** 2)


def main():
    print("A. minimum exposures for a 90% interval no wider than w")
    for w in (0.02, 0.01, 0.005):
        print("  w = {:.3f}: n >= {:,}".format(w, min_n(w)))
    print("B. sessions per arm to detect 0.040 -> p1 (alpha 0.05, 0.8)")
    for p1 in (0.044, 0.048, 0.052, 0.060):
        print("  p1 = {:.3f} (lift {:.0%}): {:,}".format(
            p1, p1 / 0.04 - 1, per_arm(0.040, p1)))
    print("C. true rate stays 0.040; chance a window looks <= 0.032")
    rng = random.Random(4301)
    for n in (100, 300, 1000, 3000):
        hit = 0
        for _ in range(4000):
            k = 0
            for _ in range(n):
                k += 1 if rng.random() < 0.04 else 0
            hit += 1 if k / n <= 0.032 else 0
        print("  window {:>5,} sessions: {:.3f}".format(n, hit / 4000))


if __name__ == "__main__":
    main()

Single source and citation

Fan, Y. (2026). Next-Generation Knowledge Distillation, Context-Parametric Compilation and Live-Conversion Feedback Calibration (Version 1.0.0), Chapter 4: Online Bayesian Calibration: Giving Every Rule a Beta Distribution That Grows. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-377 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026377,
  author    = {Fan, Yafu},
  title     = {Next-Generation Knowledge Distillation, Context-Parametric Compilation and Live-Conversion Feedback Calibration},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book