5D Library › Bayesian Search in Unknown Landscapes, Optimal Stopping and Gittins Indices · 中文

Chapter 8 Practice and a Pre-Registered Trial

Note

The checklist in this chapter is a thinking tool, not investment, start-up or career advice, and it promises no results. The example is a fictional “Qingshi Studio”; all its channels, priors, conversion rates and costs are invented to demonstrate the calculations and do not come from any real organisation or client.

Part 1 A four-step checklist for searching and stopping

The models of the first six chapters can be organised into four questions. They apply to any situation in which one searches among several uncertain options and must decide when to stop.

  1. State the search space and the prior. What are the candidates (channels, directions, people, places)? For each, what does one believe before seeing data? Write each prior as “worth so many trials” so that its weight is visible, and if necessary check whether these subjective probabilities are calibrated, using the methods of Volume 26 (Chapter 3).
  2. Set the stopping rule. Under what condition does exploration end — for example, “the posterior probability that one candidate is best reaches 0.90” or “the budget runs out”? Is the aim to earn while learning, or to learn and then earn (Chapters 4 and 6)?
  3. Allocate the exploration budget. By what rule are trials allocated among the candidates? Is there a minimum number of trials for each candidate, so that one or two early failures do not “starve” a good direction (Chapters 3 and 6)?
  4. Set the stop-loss boundary in advance. Once a direction is chosen, on what evidence will it be abandoned? Write down the break-even point, the target, the error rates and the cap on trials, and confirm that money already spent does not appear in the boundary (Chapter 5).

Calculation

The checklist applied to the fictional “Qingshi Studio”, which tests a new product through five channels, A to E, with a total budget of 1,500 trials. Step 1: the team’s priors (beta distributions) and their 90 % intervals (estimated from 20,000 draws). Steps 2–3: trials are allocated by Thompson sampling; every 50 trials the posterior probability that each channel is best is checked, and a channel is chosen when that probability reaches 0.90. Step 4: for the chosen channel, a sequential probability ratio test of a break-even conversion rate of 0.08 against a target of 0.12 sets abandonment and confirmation boundaries (α=0.05\alpha = 0.05, β=0.10\beta = 0.10). Because the example is fictional, the program finally lists the “true” conversion rates, which the team does not know.

# Four-step checklist on an invented "Qingshi Studio" testing five channels (all numbers
# made up). Step 1 priors; Steps 2-3 Thompson allocation, commit when P(best) >= 0.90;
# Step 4 stop-loss: Wald SPRT, break-even 0.08 against target 0.12.
import math, random

NAMES = ["A", "B", "C", "D", "E"]
PRIOR = [(2, 18), (2, 18), (1, 9), (3, 27), (1, 19)]     # team's stated beliefs
TRUE = [0.09, 0.13, 0.07, 0.11, 0.05]                    # unknown to the team
rng = random.Random(42)

def p_best(s, n, draws=4000):
    wins = [0] * 5
    for _ in range(draws):
        th = [rng.betavariate(a + s[i], b + n[i] - s[i])
              for i, (a, b) in enumerate(PRIOR)]
        wins[max(range(5), key=th.__getitem__)] += 1
    return [w / draws for w in wins]

def quantiles(a, b, m=20000):
    xs = sorted(rng.betavariate(a, b) for _ in range(m))
    return xs[m // 20], xs[19 * m // 20]

if __name__ == "__main__":
    print("Step 1: prior for each channel (mean, 90 % interval, weight in trials)")
    for nm, (a, b) in zip(NAMES, PRIOR):
        lo, hi = quantiles(a, b)
        print(f"   {nm}: mean {a / (a + b):.3f}, {lo:.3f} to {hi:.3f},"
              f" worth {a + b} trials")
    s, n, budget = [0] * 5, [0] * 5, 1500
    print("Steps 2-3: Thompson allocation; commit at P(best) >= 0.90 (checked every 50)")
    for t in range(1, budget + 1):
        th = [rng.betavariate(a + s[i], b + n[i] - s[i])
              for i, (a, b) in enumerate(PRIOR)]
        i = max(range(5), key=th.__getitem__)
        s[i] += rng.random() < TRUE[i]; n[i] += 1
        if t % 50 == 0:
            pb = p_best(s, n)
            if t % 300 == 0 or max(pb) >= 0.90:
                print(f"   after {t:4d}: trials {n}, P(best) "
                      + " ".join(f"{x:.2f}" for x in pb))
            if max(pb) >= 0.90:
                break
    lead = max(range(5), key=pb.__getitem__)
    print(f"   commit to {NAMES[lead]} after {t} of {budget} trials; observed rates "
          + " ".join(f"{s[i] / n[i]:.3f}" for i in range(5)))
    print("   (invented truth: " + " ".join(f"{p:.2f}" for p in TRUE) + ")")
    print("Step 4: SPRT 0.08 vs 0.12, alpha 0.05, beta 0.10; 50/100/200/400 new trials")
    la, lb = math.log(0.9 / 0.05), math.log(0.1 / 0.95)
    up, dn = math.log(0.12 / 0.08), math.log(0.88 / 0.92)
    ms = (50, 100, 200, 400)
    low = [math.floor((lb - m * dn) / (up - dn)) for m in ms]
    print("   abandon if successes <= " + ", ".join("-" if x < 0 else str(x) for x in low)
          + "; confirm if >= " + ", ".join(str(math.ceil((la - m * dn) / (up - dn)))
                                          for m in ms))
# Step 1: prior for each channel (mean, 90 % interval, weight in trials)
#    A: mean 0.100, 0.019 to 0.224, worth 20 trials
#    B: mean 0.100, 0.019 to 0.227, worth 20 trials
#    C: mean 0.100, 0.006 to 0.283, worth 10 trials
#    D: mean 0.100, 0.029 to 0.201, worth 30 trials
#    E: mean 0.050, 0.003 to 0.145, worth 20 trials
# Steps 2-3: Thompson allocation; commit at P(best) >= 0.90 (checked every 50)
#    after  300: trials [31, 47, 70, 137, 15], P(best) 0.06 0.04 0.10 0.72 0.07
#    after  600: trials [37, 53, 111, 376, 23], P(best) 0.09 0.03 0.10 0.74 0.04
#    after  900: trials [63, 57, 144, 600, 36], P(best) 0.04 0.03 0.08 0.84 0.02
#    after 1200: trials [73, 67, 161, 859, 40], P(best) 0.02 0.03 0.03 0.92 0.01
#    commit to D after 1200 of 1500 trials; observed rates 0.041 0.045 0.068 0.116 0.025
#    (invented truth: 0.09 0.13 0.07 0.11 0.05)
# Step 4: SPRT 0.08 vs 0.12, alpha 0.05, beta 0.10; 50/100/200/400 new trials
#    abandon if successes <= -, 4, 14, 34; confirm if >= 12, 17, 27, 46

Step 1: A, B, C and D all have prior mean 0.10 but different weights: D is worth 30 trials, with a 90 % interval from 0.029 to 0.201; C is worth only 10, with an interval as wide as 0.006 to 0.283. E has prior mean 0.05. (A and B have the same prior; the slight difference in their intervals is sampling error.)

Steps 2–3: in the first 300 trials, D receives 137 and its posterior probability of being best is 0.72; by trial 1,200, D has received 859 trials and the probability reaches 0.92, so the team chooses D with 300 trials of budget to spare. Among the observed conversion rates, D’s is highest (0.116) and B’s only 0.045.

But in this fictional example the true best channel is B (0.13), not D (0.11). B was unlucky early on, converting only 3 times in 67 trials; Thompson sampling then moved trials to other channels and B never had a chance to recover. This is not a bug: a posterior probability of 0.92 means a chance of about 8 % of choosing wrongly, and this run happened to fall within that 8 %. It shows two additions that Steps 2 and 3 need: first, a minimum number of trials for each candidate, or the top-two Thompson sampling of Chapter 6, which keeps some trials for challengers; second, a check before committing as to whether any eliminated candidate was dropped on the strength of only a few early failures (the “premature exclusion” of Chapter 3).

Step 4: for the chosen channel D, there is no abandonment within the first 50 new trials, and 12 conversions confirm it; at 100 trials, abandon with at most 4 and confirm with 17; at 200 trials, 14 and 27; at 400 trials, 34 and 46. The boundaries depend only on the break-even point, the target and the error rates, not on how much has already been spent on D.

Part 2 A pre-registered randomised controlled trial

Question

In a sequential-choice task with relative ranks only (the classical secretary problem, with 40 virtual applicants per game), does learning a stopping-threshold rule in advance (“let the first 15 pass, then choose the first one better than everyone before”), compared with control instructions that teach only the rules of the task, raise the share of games in which the overall best is chosen?

Evidence

Existing evidence. The studies described in Chapter 2 show that in the classical task with relative ranks only, participants tend to stop earlier than the optimal rule (Seale and Rapoport 1997); that in repeated games in which the distribution can be learned, players approach the optimal threshold after a few games (Goldstein et al. 2020); and, as Bearden pointed out, that when payoff is actual quality, stopping earlier is in fact optimal (Bearden 2006). But the author found no pre-registered trial in the classical task that randomly compared “learning a threshold rule in advance” with control instructions, with the share of games in which the best is chosen as the single primary outcome.

Design

Element Specification
Design Two-arm parallel online trial, individually randomised, stratified by self-reported prior study of the secretary problem, 1:1
Participants Adults aged 18 or over who read the task language fluently and work on a computer, recruited through a public online research-participant platform; not recruited from the author’s community, students or customers, including the clients and students of the author’s businesses
Exclusion Previous participation in the pilot; failing both attention checks (after randomisation; counts towards the “excluded share”)
Task 20 games of the classical secretary problem with 40 virtual applicants, relative ranks only, no recall; a bonus for each game in which the overall best is hired
Intervention and control An interactive explanation of about 8 minutes each, plus 3 practice games; see below
Primary outcome The share of the 20 games in which each participant chose the overall best
Secondary outcomes Share of games in which the last applicant was hired by default (“ending empty-handed”), mean stopping position, mean rank of the person hired, fitted individual threshold, change from the first 10 games to the last 10
Sample size 190 analysable and 224 recruited per arm, 448 in all
Analysis Linear regression of the primary outcome on arm and stratum; difference in means dd (intervention minus control) with 95 % CI, two-sided α=0.05\alpha = 0.05; sensitivity analysis by logistic regression on each game with participant as a random effect

Task. In each game 40 virtual applicants appear one at a time in random order, and the screen shows only “this applicant’s rank among those seen so far”; the participant chooses “hire” or “pass” on the spot, an applicant passed cannot be recalled, and if nobody has been hired by the last applicant, that applicant is hired automatically. A bonus is paid in each game in which the person hired is the overall best, and nothing otherwise, identically in both arms; this scoring is explained to both arms in advance, making “only the best counts” an explicit objective.

The intervention arm’s explanation covers what the threshold rule is (with 40 applicants, let the first 15 pass), why its success probability is about 37 % and why that is already the best possible; it is followed by 3 practice games, each ending with a display of where the overall best was and what the rule would have done, and by 3 multiple-choice questions on the rule (the comprehension check). The control arm’s explanation covers only the interface, the rules, the scoring and the remark that choosing the best is inherently hard, with no strategy; it is followed by the same 3 practice games, each ending with a display only of where the overall best was, and 3 questions on the interface. Participants do not know the hypothesis or the other arm’s content; randomisation and outcome recording are automatic; analysts do not know what the arm labels mean until the analysis plan is locked. Participants receive a fixed fee plus the per-game bonuses.

There is a single primary outcome. The secondary outcomes help to interpret the result, in particular the hypothesis of Chapter 2: if the intervention arm chooses the best more often, does its share of games “ending empty-handed” rise at the same time? Secondary outcomes are reported only as estimates with intervals and play no part in the primary decision.

Sample size

Calculation

# Chapter 8 trial. Part 1: a MODEL (simulated participants, not data). 20 games of the
# secretary problem, n = 40, ranks only; each person skips round(40 x) then takes the
# first relative best; control x ~ N(0.22, 0.10), trained x ~ N(0.33, 0.07), truncated
# to [0.02, 0.90]: assumptions, not estimates. Outcome: share of games best chosen.
# Part 2: sample size for a difference in mean shares, two-sided alpha 0.05.
import math, random
from statistics import NormalDist, mean, stdev

N, GAMES = 40, 20

def p_success(r):                        # skip r - 1 applicants, exact probability
    return 1 / N if r == 1 else (r - 1) / N * sum(1 / (i - 1) for i in range(r, N + 1))

def participant(mu, sd, rng):
    x = min(0.90, max(0.02, rng.gauss(mu, sd)))
    p = p_success(round(N * x) + 1)
    return sum(rng.random() < p for _ in range(GAMES)) / GAMES

def n_per_arm(sd, diff, power=0.90, alpha=0.05):
    z = NormalDist().inv_cdf
    return math.ceil(2 * (z(1 - alpha / 2) + z(power)) ** 2 * sd ** 2 / diff ** 2)

if __name__ == "__main__":
    rng = random.Random(1997)
    ctrl = [participant(0.22, 0.10, rng) for _ in range(20000)]
    trn = [participant(0.33, 0.07, rng) for _ in range(20000)]
    print(f"model: control mean share {mean(ctrl):.3f} (SD {stdev(ctrl):.3f}),"
          f" trained {mean(trn):.3f} (SD {stdev(trn):.3f})")
    print(f"model difference {mean(trn) - mean(ctrl):+.3f};"
          f" ceiling with the optimal cutoff: {p_success(16):.3f}")
    print("analysable participants per arm, power 90 %:")
    print("   SD of share   diff 0.03   diff 0.04   diff 0.05")
    for sd in (0.10, 0.12, 0.14):
        cols = "      ".join(f"{n_per_arm(sd, d):6d}" for d in (0.03, 0.04, 0.05))
        print(f"      {sd:.2f}       {cols}")
    print("pre-registered plan: SD 0.12, smallest effect of interest 0.04, 15 % excluded")
    n = n_per_arm(0.12, 0.04); rec = math.ceil(n / 0.85)
    print(f"   power 90 %: {n} analysable and {rec} recruited per arm, {2 * rec} in all")
# model: control mean share 0.321 (SD 0.120), trained 0.367 (SD 0.107)
# model difference +0.046; ceiling with the optimal cutoff: 0.376
# analysable participants per arm, power 90 %:
#    SD of share   diff 0.03   diff 0.04   diff 0.05
#       0.10          234         132          85
#       0.12          337         190         122
#       0.14          458         258         165
# pre-registered plan: SD 0.12, smallest effect of interest 0.04, 15 % excluded
#    power 90 %: 190 analysable and 224 recruited per arm, 448 in all

The first part of the program is not data but a model of the hypothesis: each participant uses a personal threshold; the control arm’s threshold share averages 0.22 (stopping early, an assumption chosen to follow the direction of earlier studies, not an estimate from any study) and the intervention arm’s 0.33. In the model the control arm chooses the best in a mean share of 0.321 of games and the intervention arm in 0.367, a difference of 0.046; even if everyone used the optimal threshold exactly, the share would be only 0.376. The effect this trial can detect therefore has a ceiling: if the control arm already does reasonably well (say 0.34), no training can raise the share by 4 percentage points.

The sample size rests on the following assumptions: a smallest effect of interest of 4 percentage points (0.04); a standard deviation of each participant’s share of 0.12 (the binomial sampling error over 20 games is about 0.107, plus differences between people; the model’s standard deviations are 0.107 to 0.120); two-sided α=0.05\alpha = 0.05; and power of 90 %. Each arm needs 190 analysable participants; allowing for about 15 % excluded for not finishing or failing the attention checks, each arm recruits 224, 448 in all. With a standard deviation of 0.10 or 0.14, each arm would need 132 or 258; with a smallest effect of interest of 0.03 and standard deviation 0.12, 337. All these parameters are assumptions. Before the main trial, a small pilot checks only that the task software, the length of the explanations and the attention checks work properly; it is not used to change the sample size.

Note

What “90 % power” means. As in Volume 33, 90 % is the probability of reaching some “benefit” conclusion when the true effect equals the smallest effect of interest (d=0.04d = 0.04); in that case the chance that the decision rule reaches the strongest conclusion, “an improvement of at least 4 percentage points”, is only about 2.5 %. The decision program below checks this by simulation.

Decision rule

Calculation

# Decision rule for the Chapter 8 trial, fixed before any data are collected.
# d = mean share of games won (best applicant chosen), trained arm minus control arm,
# with its 95 % CI. Smallest effect of interest (SESOI): 0.04 (4 percentage points).
# Gates: more than 20 % of randomised participants excluded (failed attention checks,
# unfinished), or fewer than 70 % of the trained arm passing the comprehension check
# -> NOT INTERPRETABLE.
import math, random
from statistics import NormalDist
SESOI = 0.04

def verdict(d, lo, hi, excluded=0.10, understood=0.85):
    if excluded > 0.20:
        return (f"d = {d:+.3f}: NOT INTERPRETABLE - {100 * excluded:.0f} % excluded"
                " (limit 20 %)")
    if understood < 0.70:
        return (f"d = {d:+.3f}: NOT INTERPRETABLE - only {100 * understood:.0f} %"
                " understood the rule")
    if lo >= SESOI:      v = "training helps by at least 4 points"
    elif lo > 0:         v = "training helps; 4 points not shown"
    elif hi < 0:         v = "training HARMS"
    elif lo >= -SESOI and hi <= SESOI: v = "no difference worth acting on"
    else:                v = "unclear: the interval is too wide to decide"
    return f"d = {d:+.3f} ({lo:+.3f} to {hi:+.3f}): {v}"

def label(lo, hi):
    return verdict(0, lo, hi).split(": ", 1)[1]

if __name__ == "__main__":
    for case in ((0.071, 0.047, 0.095), (0.046, 0.022, 0.070), (0.004, -0.020, 0.028),
                 (-0.031, -0.055, -0.007), (0.020, -0.012, 0.052),
                 (0.060, 0.036, 0.084, 0.26), (0.050, 0.026, 0.074, 0.08, 0.62)):
        print(verdict(*case))
    # How often does each verdict occur? 190 analysable per arm, SD of the share 0.12,
    # 10,000 simulated trials for each true d.
    rng = random.Random(42); z = NormalDist().inv_cdf(0.975); n, sd = 190, 0.12
    for true in (0.04, 0.06, 0.0):
        tally = {}
        for _ in range(10000):
            a = [rng.gauss(true, sd) for _ in range(n)]
            b = [rng.gauss(0.0, sd) for _ in range(n)]
            ma, mb = sum(a) / n, sum(b) / n
            ss = sum((x - ma) ** 2 for x in a) + sum((x - mb) ** 2 for x in b)
            d, se = ma - mb, math.sqrt(ss / (2 * n - 2) * (2 / n))
            v = label(d - z * se, d + z * se)
            tally[v] = tally.get(v, 0) + 1
        short = {"training helps by at least 4 points": "helps >= 4",
                 "training helps; 4 points not shown": "helps", "training HARMS": "harms",
                 "no difference worth acting on": "no difference",
                 "unclear: the interval is too wide to decide": "unclear"}
        ranked = sorted(tally.items(), key=lambda kv: -kv[1])
        print(f"true d = {true:+.2f}: " + ", ".join(f"{short[k]} {c / 100:.1f} %"
                                                  for k, c in ranked))
# d = +0.071 (+0.047 to +0.095): training helps by at least 4 points
# d = +0.046 (+0.022 to +0.070): training helps; 4 points not shown
# d = +0.004 (-0.020 to +0.028): no difference worth acting on
# d = -0.031 (-0.055 to -0.007): training HARMS
# d = +0.020 (-0.012 to +0.052): unclear: the interval is too wide to decide
# d = +0.060: NOT INTERPRETABLE - 26 % excluded (limit 20 %)
# d = +0.050: NOT INTERPRETABLE - only 62 % understood the rule
# true d = +0.04: helps 86.9 %, unclear 7.9 %, no difference 2.7 %, helps >= 4 2.6 %
# true d = +0.06: helps 63.3 %, helps >= 4 36.5 %, unclear 0.2 %
# true d = +0.00: no difference 80.1 %, unclear 14.5 %, harms 2.9 %, helps 2.4 %

The first seven example results are invented. If more than 20 % of randomised participants are excluded, or fewer than 70 % of the intervention arm pass the comprehension check, the report says “not interpretable” whatever the result. Only if the lower limit of the 95 % confidence interval for dd is at least 0.04 does the report say “training raises the share by at least 4 percentage points”. The simulation shows that with 190 per arm and a true dd of exactly 0.04, 86.9 % of 10,000 simulated trials conclude “an improvement, but 4 points not shown” and 2.6 % reach the strongest conclusion, about 90 % in all; with a true dd of 0.06, 36.5 % reach the strongest conclusion; with no true effect, 80.1 % conclude “no difference worth acting on”, 2.4 % wrongly conclude “an improvement” and 2.9 % “training harms”. Secondary outcomes are reported only as estimates with intervals and are not used to overturn the primary conclusion.

What this trial cannot show

First, the task is scored as “only the best counts” in order to test the rule under its own objective; even a positive result does not show that learning the 37 % rule helps in real hiring, choosing a partner or investing. Chapter 6 showed that when payoff is actual quality or interviews are costly, the 37 % rule is far from optimal, and applying it may do harm. Second, this is an abstract online task with small bonuses and virtual applicants; the emotions, time pressure and social factors of real decisions are absent. Third, the intervention combines the rule, an explanation and practice with feedback, and a positive result cannot tell which part worked. Fourth, the outcome is measured immediately after the explanation, so it cannot show how long any effect lasts. Fifth, the possible effect is capped by a ceiling of about 0.376, so even if training “works”, the absolute improvement is small. Sixth, participants on online platforms represent neither the general population nor experienced recruiters.

Ethics and conflicts of interest

This is a low-risk study: the task is an abstract choice game, involves no health intervention, and collects no health information or identifying personal data. Even so, the trial must be reviewed and approved by an independent ethics committee, every participant gives informed consent online, and the protocol and analysis plan are registered before the first participant is enrolled (Nosek et al. 2018). The task instructions state the scoring and payment truthfully; withholding the study hypothesis is non-disclosure, not deception, and at the end everyone is told the purpose of the study and the control arm is offered the intervention arm’s explanation. Anyone may withdraw at any time and is paid for the part completed. Data are stored anonymously and used only to compute the pre-specified measures. The results will be published whether positive or negative.

Disclosure

The author uses the works of 5D Library, including this book, as the methodological basis of his community courses, the 5D Ascension Cards (Dimensional Shift Card Decks) and the Yafu AI knowledge base, and therefore has a commercial interest in the subject of this book. The author also founded and runs Asia Influencer X and Yafu Global, whose business involves screening influencers and talent and testing new lanes and new projects, and therefore has a direct commercial interest in the subject of this book. The book promotes no course, card deck or service, is not investment, start-up or career advice, and uses no client data from these businesses; the Chapter 8 trial is designed to be run and analysed by researchers independent of the author’s business and does not recruit from the author’s community, students or customers, including the clients and students of these businesses; all the code and the study protocol are published openly for anyone to check. All models in this book are offered solely for discussion of mathematics and statistical decision theory, and the book recommends no specific project, lane or candidate.

References

Bearden, J. Neil. 2006. “A New Secretary Problem with Rank-Based Selection and Cardinal Payoffs.” Journal of Mathematical Psychology 50: 58–59.
Goldstein, Daniel G., R. Preston McAfee, Siddharth Suri, and James R. Wright. 2020. “Learning When to Stop Searching.” Management Science 66 (3): 1375–94. https://doi.org/10.1287/mnsc.2018.3245.
Nosek, Brian A., Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor. 2018. “The Preregistration Revolution.” Proceedings of the National Academy of Sciences 115 (11): 2600–2606. https://doi.org/10.1073/pnas.1708274114.
Seale, Darryl A., and Amnon Rapoport. 1997. “Sequential Decision Making with Relative Ranks: An Experimental Investigation of the ‘Secretary Problem’.” Organizational Behavior and Human Decision Processes 69 (3): 221–36.

Fan, Y. (2026). Bayesian Search in Unknown Landscapes, Optimal Stopping and Gittins Indices: Exploration–Exploitation Trade-offs and Sunk-Cost Truncation in Statistical Decision Theory [未知地形中的贝叶斯搜索、秘书问题与吉廷斯指数] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23810102 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).