What does First-Move Probes and Hidden Market Truth conclude, in numbers, about “Exploration and Exploitation: The Regret Ledger of Thompson Sampling and Fixed Habits”?

Answer: Under Thompson sampling, each doubling of the horizon adds a roughly constant amount of regret: 233.8, 296.7, 280.4 and 260.4. That is what logarithmic growth looks like, of the same order as the Lai-Robbins bound. Exploring for 200 visitors adds ever larger amounts: 173.2, 320.8, 564.7 and 1,120.2, because the stuck runs keep losing money in proportion to time. Exploring for 800 visitors loses heavily at first, reaching 1,594.4 after 1,000 visitors; afterwards it adds little, but at 8,000 visitors its total of 1,919.7 is still higher than the 1,604.7 of Thompson sampling.

Scope of the numbers: Parameters are illustrative, calculations come from the book's programs, and all data are synthetic. This is general business analysis, not personalised legal, tax or investment advice; it promises no income.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 1000, 1120.2, 1594.4, 1604.7, 173.2, 1919.7

mean regret at each horizon (and added since the last one)
policy5001,0002,0004,0008,000
etc200542.4715.61,036.41,601.12,721.3
added542.4173.2320.8564.71,120.2
etc800975.01,594.41,668.41,759.71,919.7
added975.0619.474.091.4160.0
thompson533.4767.21,063.91,344.31,604.7
added533.4233.8296.7280.4260.4
Show the full program
# Chapter 4. p42: how regret grows when the horizon doubles.
# Same four illustrative tiers as p41. A policy that keeps learning adds
# roughly the same regret per doubling of the horizon (logarithmic
# growth, the Lai-Robbins order); a policy that stops learning after a
# fixed explore phase adds regret in proportion to time whenever its
# early pick was wrong. 60 runs to 8,000 visitors.
import math
import random

PRICES = [20, 30, 40, 50]
CONV = [0.30, 0.24, 0.12, 0.06]
BEST = max(p * c for p, c in zip(PRICES, CONV))
MARKS = (500, 1000, 2000, 4000, 8000)


def play(rng, policy, horizon):
    buys, shows = [0] * 4, [0] * 4
    lost, out = [], {}
    explore = {"etc200": 200, "etc800": 800}.get(policy, 0)
    for t in range(horizon):
        if t < explore:
            a = t % 4
        elif explore:
            est = [PRICES[i] * (buys[i] + 1) / (shows[i] + 2)
                   for i in range(4)]
            a = est.index(max(est))
        else:
            draw = [PRICES[i] * rng.betavariate(1 + buys[i],
                                                1 + shows[i] - buys[i])
                    for i in range(4)]
            a = draw.index(max(draw))
        shows[a] += 1
        if rng.random() < CONV[a]:
            buys[a] += 1
        lost.append(BEST - PRICES[a] * CONV[a])
        if t + 1 in MARKS:
            out[t + 1] = math.fsum(lost)
    return out


if __name__ == "__main__":
    rng = random.Random(39542)
    reps = 60
    print("mean regret at each horizon (and added since the last one)")
    print("  policy  " + "".join("{:>13,}".format(m) for m in MARKS))
    for pol in ("etc200", "etc800", "thompson"):
        acc = {m: [] for m in MARKS}
        for _ in range(reps):
            out = play(rng, pol, MARKS[-1])
            for m in MARKS:
                acc[m].append(out[m])
        means = [math.fsum(acc[m]) / reps for m in MARKS]
        print("  {:<8}".format(pol)
              + "".join("{:>13,.1f}".format(v) for v in means))
        adds = [means[0]] + [b - a for a, b in zip(means, means[1:])]
        print("  {:<8}".format("  added")
              + "".join("{:>13,.1f}".format(v) for v in adds))

Single source and citation

Fan, Y. (2026). High-Agency Active Inference, Intentional Perturbation Probes and Hidden Market-Truth Exploration (Version 1.0.0), Chapter 4: Exploration and Exploitation: The Regret Ledger of Thompson Sampling and Fixed Habits. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-395 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026395,
  author    = {Fan, Yafu},
  title     = {High-Agency Active Inference, Intentional Perturbation Probes and Hidden Market-Truth Exploration},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book