What does First-Move Probes and Hidden Market Truth conclude, in numbers, about “Exploration and Exploitation: The Regret Ledger of Thompson Sampling and Fixed Habits”?
Answer: Under Thompson sampling, each doubling of the horizon adds a roughly constant amount of regret: 233.8, 296.7, 280.4 and 260.4. That is what logarithmic growth looks like, of the same order as the Lai-Robbins bound. Exploring for 200 visitors adds ever larger amounts: 173.2, 320.8, 564.7 and 1,120.2, because the stuck runs keep losing money in proportion to time. Exploring for 800 visitors loses heavily at first, reaching 1,594.4 after 1,000 visitors; afterwards it adds little, but at 8,000 visitors its total of 1,919.7 is still higher than the 1,604.7 of Thompson sampling.
Scope of the numbers: Parameters are illustrative, calculations come from the book's programs, and all data are synthetic. This is general business analysis, not personalised legal, tax or investment advice; it promises no income.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 1000, 1120.2, 1594.4, 1604.7, 173.2, 1919.7
| mean regret at each horizon (and added since the last one) | |||||
| policy | 500 | 1,000 | 2,000 | 4,000 | 8,000 |
| etc200 | 542.4 | 715.6 | 1,036.4 | 1,601.1 | 2,721.3 |
| added | 542.4 | 173.2 | 320.8 | 564.7 | 1,120.2 |
| etc800 | 975.0 | 1,594.4 | 1,668.4 | 1,759.7 | 1,919.7 |
| added | 975.0 | 619.4 | 74.0 | 91.4 | 160.0 |
| thompson | 533.4 | 767.2 | 1,063.9 | 1,344.3 | 1,604.7 |
| added | 533.4 | 233.8 | 296.7 | 280.4 | 260.4 |
Show the full program
# Chapter 4. p42: how regret grows when the horizon doubles.
# Same four illustrative tiers as p41. A policy that keeps learning adds
# roughly the same regret per doubling of the horizon (logarithmic
# growth, the Lai-Robbins order); a policy that stops learning after a
# fixed explore phase adds regret in proportion to time whenever its
# early pick was wrong. 60 runs to 8,000 visitors.
import math
import random
PRICES = [20, 30, 40, 50]
CONV = [0.30, 0.24, 0.12, 0.06]
BEST = max(p * c for p, c in zip(PRICES, CONV))
MARKS = (500, 1000, 2000, 4000, 8000)
def play(rng, policy, horizon):
buys, shows = [0] * 4, [0] * 4
lost, out = [], {}
explore = {"etc200": 200, "etc800": 800}.get(policy, 0)
for t in range(horizon):
if t < explore:
a = t % 4
elif explore:
est = [PRICES[i] * (buys[i] + 1) / (shows[i] + 2)
for i in range(4)]
a = est.index(max(est))
else:
draw = [PRICES[i] * rng.betavariate(1 + buys[i],
1 + shows[i] - buys[i])
for i in range(4)]
a = draw.index(max(draw))
shows[a] += 1
if rng.random() < CONV[a]:
buys[a] += 1
lost.append(BEST - PRICES[a] * CONV[a])
if t + 1 in MARKS:
out[t + 1] = math.fsum(lost)
return out
if __name__ == "__main__":
rng = random.Random(39542)
reps = 60
print("mean regret at each horizon (and added since the last one)")
print(" policy " + "".join("{:>13,}".format(m) for m in MARKS))
for pol in ("etc200", "etc800", "thompson"):
acc = {m: [] for m in MARKS}
for _ in range(reps):
out = play(rng, pol, MARKS[-1])
for m in MARKS:
acc[m].append(out[m])
means = [math.fsum(acc[m]) / reps for m in MARKS]
print(" {:<8}".format(pol)
+ "".join("{:>13,.1f}".format(v) for v in means))
adds = [means[0]] + [b - a for a, b in zip(means, means[1:])]
print(" {:<8}".format(" added")
+ "".join("{:>13,.1f}".format(v) for v in adds))Single source and citation
Fan, Y. (2026). High-Agency Active Inference, Intentional Perturbation Probes and Hidden Market-Truth Exploration (Version 1.0.0), Chapter 4: Exploration and Exploitation: The Regret Ledger of Thompson Sampling and Fixed Habits. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-395 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026395,
author = {Fan, Yafu},
title = {High-Agency Active Inference, Intentional Perturbation Probes and Hidden Market-Truth Exploration},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Bias Floor of Confounding: Why a Longer Passive Log Is Wrong with More Confidence
- Chapter 2 · Identifiability: What the Back Door, the Instrument and the Published Threshold Each Require
- Chapter 3 · Expected Information Gain: Put the Probe Where It Teaches the Most
- Chapter 5 · Commitment Strength Made Operational: Probes per Week, Delay to the First Probe and the Stopping Rule
- Chapter 6 · Bringing Willingness to Pay into View, and a Counterpart's Floor: BDM, Hypothetical Bias and Inference from Public Information
- Chapter 7 · The Classics Examined: Lu You, Wang Yangming, Bacon and Bernard on Doing and on Provoked Observation
- Chapter 8 · Field Tools and Checker Specifications: The First-Move Probe Ledger, Pre-Registered Tests and the Plan for Refutation