5D Library › Books › 5DL-B-042

未知地形中的贝叶斯搜索、秘书问题与吉廷斯指数

探索—利用权衡与沉没成本及时截断的统计决策论

Bayesian Search in Unknown Landscapes, Optimal Stopping and Gittins Indices: Exploration–Exploitation Trade-offs and Sunk-Cost Truncation in Statistical Decision Theory

  • 范雅福 Fan Yafu
  • Version 1.0.0 · 2026-10-30
  • CC-BY-NC-ND-4.0
  • 5DL-B-042
DOI10.5281/zenodo.23810102

摘要

先看过约 37% 的人而不录用、之后录用第一个比之前所有人都好的人,选中最佳者的概率约为 37%——这条「37% 法则」流传很广,但它成立的条件相当苛刻。本书用统计决策论考察三类相关的问题:什么时候停止观察并行动(停时),往哪里找(贝叶斯搜索),以及为了信息愿意付出多少眼前的代价(探索—利用权衡)。依次讨论:秘书问题在有限 n 时的精确最优阈值与成功概率(n = 10 时跳过 3 人、成功概率 0.399,n = 100 时 0.371,极限 1/e;最优点附近很平坦,n = 40 时有 37.5% 的情形空手而归),弗格森 1989 年的考证(加德纳的原始游戏是两人博弈,开普勒的故事违反「不能回头」);完全信息版本(约 0.58)、最小期望名次(极限约 3.8695,而 37% 规则在 n = 100 时为 19.6)、录用可能被拒绝、可以回头与人数未知(最佳阈值约 13.5%,成功概率约 0.27)等变体,以及实验中人们停得太早与在可学习的重复游戏中迅速接近最优;库普曼的搜索论、贝叶斯更新与「后验概率 × 发现概率」的贪婪规则,「天蝎号」与法航 447 的残骸搜寻(后者的关键在于重新检查信标是否工作的假设,联合模型在示意网格中对两种真相都稳健),以及觅食生态学中查诺夫的边际价值定理(路程越远停留越久,离开时斑块并未吃空)与列维飞行觅食假说之争(μ = 2 只在目标稀疏、可重复访问时近似最优,一维模型中目标越稀疏最优值越接近 2,一次性目标时近直线运动更好;1996 年信天翁的幂律在 2007 年的重新分析中消失);吉廷斯指数的校准法与重启法计算(Beta(1, 1)、折扣 0.9 时为 0.7029)及其成立条件(折扣 0.9 的两臂问题中贪婪策略只少约 0.3%),汤普森抽样、UCB1 与莱—罗宾斯渐近下界(有限 T 下汤普森抽样的后悔可低于 c ln T);沉没成本的证据与争议(阿克斯与布卢默 1985、罗特等人 2015 年元分析的合并效应约 0.5、多为假想情境,但项目推进类决策的现场研究中效应更大;有真实激励的实验室实验中效应小而不稳定),单臂老虎机中「理性的坚持」与「谬误的坚持」的区别(示意参数下短视的过早放弃损失约 29%,最优规则加上沉没成本倾向损失约 6%),以及沃尔德序贯概率比检验给出的预设止损边界;赛道测试中边学边赚与先学后赚两种目标,以及人才筛选中按实际水平计分时最佳阈值约为 √n。

八章与《孙子兵法·作战篇》「兵贵胜,不贵久」、《三国志·魏书·郭嘉传》「兵贵神速」(不出自《孙子兵法》)、《九变篇》「途有所不由」(框架作「涂」,已注明)、《军争篇》以及《韩非子·显学》「试之官职,课其功伐」与《二柄》「审合刑名」(今本作「言异事也」,已注明)并读,并明确古代军事与统治经验与现代停时理论的界限;最后给出一份四步搜索与停止核查表,以及一项比较「事先学习停止阈值规则」与对照说明的预注册在线随机对照试验方案(任务为 20 局各 40 名虚拟求职者的经典秘书问题,唯一主要结局为选中全体最佳者的局数比例,最小关注效应 4 个百分点,把握度 90%,每组 190 人可分析、招募 224 人,判定规则写成代码)。书中以标注框区分已发表的证据、可运行的 Python 计算(17 段程序,全部核对)、原典与作者的假说。本书是教学与研究用的数学模型,不是投资、创业、职业或人力资源建议。

关键词:秘书问题;最优停止;1/e 法则;完全信息秘书问题;期望名次;贝叶斯搜索论;搜索与发现;法航 447;多臂老虎机;探索—利用权衡;吉廷斯指数;汤普森抽样;UCB1;莱—罗宾斯下界;最佳臂识别;沉没成本效应;承诺升级;单臂老虎机;序贯概率比检验;边际价值定理;列维飞行觅食;孙子兵法;韩非子

Abstract

Look at about 37 % of the applicants without hiring, then hire the first one better than everyone before, and you choose the best with probability about 37 %: this "37 % rule" is widely repeated, but the conditions under which it holds are demanding. This book uses statistical decision theory to examine three related questions: when to stop observing and act (stopping times), where to look (Bayesian search), and how much immediate cost to accept for the sake of information (the exploration–exploitation trade-off). It covers the exact optimal threshold and success probability of the secretary problem for finite n (skip 3 of 10 with success probability 0.399; 0.371 for n = 100; limit 1/e; flat near the optimum, and for n = 40 the rule ends with nobody in 37.5 % of cases) and Ferguson's 1989 history (Gardner's original game is a two-person game, and Kepler's story violates "no recall"); variants with full information (about 0.58), minimal expected rank (limit about 3.8695, against 19.6 for the 37 % rule at n = 100), offers that may be refused, recall and an unknown number of applicants (best threshold about 13.5 %, success probability about 0.27), and how people stop too early in experiments yet quickly approach the optimum in repeated games whose distribution can be learned; Koopman's search theory, Bayesian updating and the greedy rule "posterior probability × detection probability", and the searches for the wreckage of *Scorpion* and AF 447 (where the key step was re-examining the assumption that the beacons worked, and a joint model is robust to both truths on an illustrative grid), together with Charnov's marginal value theorem from foraging ecology (the farther the travel, the longer the stay, and a patch is left before it is exhausted) and the dispute over the Lévy-flight foraging hypothesis (μ = 2 is near-optimal only for sparse, revisitable targets, the optimum in a one-dimensional model approaches 2 as targets become sparser, straight-line motion is better for one-off targets, and the albatross power law of 1996 vanished in a 2007 re-analysis); the Gittins index computed by calibration and by restart-in-state (0.7029 for Beta(1, 1) at discount 0.9) and the conditions for its optimality (in a two-armed problem with discount 0.9 a greedy policy loses only about 0.3 %), Thompson sampling, UCB1 and the asymptotic lower bound of Lai and Robbins (at finite T the regret of Thompson sampling can lie below c ln T); the evidence and disputes on sunk costs (Arkes and Blumer 1985; a pooled effect of about 0.5 in the meta-analysis of Roth and colleagues, mostly from hypothetical scenarios but larger in field studies of project decisions; small and erratic effects in laboratory experiments with real incentives), the difference between rational and fallacious persistence in a one-armed bandit (with illustrative parameters, myopic early quitting loses about 29 % and adding a sunk-cost tendency to the optimal rule about 6 %), and stop-loss boundaries set in advance by Wald's sequential probability ratio test; the two objectives of earning while learning and learning then earning in testing start-up lanes, and a best cutoff of about √n when selection is scored by actual quality.

The eight chapters are read beside "in war victory is prized, not duration" from the "Waging War" chapter of the *Sunzi*, "speed is what the army prizes" from the biography of Guo Jia in the *Records of the Three Kingdoms* (not from the *Sunzi*), "there are roads one does not follow" from the "Nine Variations" (the framework's 涂 noted against the received 途), the "Manoeuvring" chapter, and "try them in office and assess their accomplishments" from the "Eminent Teachings" and "examining whether forms and names tally" from the "Two Handles" of the *Han Feizi* (the received reading 言异事也 noted), stating where classical military and political experience ends and the modern theory of stopping begins. They end with a four-step checklist for searching and stopping and a pre-registered online randomised controlled trial comparing "learning a stopping-threshold rule in advance" with control instructions (20 games of the classical secretary problem with 40 virtual applicants each; single primary outcome the share of games in which the overall best is chosen; smallest effect of interest 4 percentage points; 90 % power; 190 analysable and 224 recruited per arm; decision rule written as code). Marked boxes separate published evidence, runnable Python calculations (17 programs, all checked), classical sources and the author's hypotheses. The book is a teaching and research text on mathematical models, not investment, start-up, career or human-resources advice.

Keywords: secretary problem; optimal stopping; 1/e rule; full-information secretary problem; expected rank; Bayesian search theory; search and detection; Air France 447; multi-armed bandit; exploration-exploitation trade-off; Gittins index; Thompson sampling; UCB1; Lai-Robbins lower bound; best-arm identification; sunk-cost effect; escalation of commitment; one-armed bandit; sequential probability ratio test; marginal value theorem; Lévy flight foraging; Sunzi; Han Feizi

目录 · Contents

  • 序:什么时候停下,往哪里找,还要不要继续English 中文Preface: When to Stop, Where to Look, and Whether to Go On
  • 第一章 秘书问题与 1/e:只看名次、不能回头时何时停下English 中文Chapter 1 The Secretary Problem and 1/e: When to Stop If You See Only Ranks and Cannot Go Back
  • 第二章 秘书问题的变体与人的停止行为English 中文Chapter 2 Variants of the Secretary Problem and How People Stop
  • 第三章 贝叶斯搜索:往哪里找,没找到又说明了什么English 中文Chapter 3 Bayesian Search: Where to Look, and What Not Finding Means
  • 第四章 多臂老虎机:吉廷斯指数、汤普森抽样与后悔下界English 中文Chapter 4 Multi-Armed Bandits: Gittins Indices, Thompson Sampling and the Regret Lower Bound
  • 第五章 沉没成本:理性的坚持、谬误的坚持与预设的止损边界English 中文Chapter 5 Sunk Costs: Rational Persistence, Fallacious Persistence and Stop-Loss Boundaries Set in Advance
  • 第六章 应用:赛道测试与人才筛选English 中文Chapter 6 Applications: Testing Lanes and Selecting People
  • 第七章 原典并读:兵贵胜不贵久、途有所不由与试之官职English 中文Chapter 7 Reading the Classics: Victory Not Duration, Roads Not Taken, and Trial in Office
  • 第八章 实践与预注册试验English 中文Chapter 8 Practice and a Pre-Registered Trial

How to cite · 引用

Fan, Y. (2026). Bayesian Search in Unknown Landscapes, Optimal Stopping and Gittins Indices: Exploration–Exploitation Trade-offs and Sunk-Cost Truncation in Statistical Decision Theory [未知地形中的贝叶斯搜索、秘书问题与吉廷斯指数] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23810102

BibTeX
@book{fan2026bayesian,
  title = {{Bayesian Search in Unknown Landscapes, Optimal Stopping and Gittins Indices}: Exploration–Exploitation Trade-offs and Sunk-Cost Truncation in Statistical Decision Theory},
  author = {Fan, Yafu},
  year = {2026},
  publisher = {5D Library},
  edition = {Version 1.0.0},
  doi = {10.5281/zenodo.23810102},
  url = {https://doi.org/10.5281/zenodo.23810102},
  note = {Chinese title: 未知地形中的贝叶斯搜索、秘书问题与吉廷斯指数}
}
DOIhttps://doi.org/10.5281/zenodo.23810102
DOI(全部版本 all versions)https://doi.org/10.5281/zenodo.23810101
ORCIDhttps://orcid.org/0009-0004-3519-1203
5D Library ID5DL-B-042
版本 Version1.0.0 · 中英双语版 Bilingual edition
出版日期 Date2026-10-30
网址 Onlinehttps://5dlibrary.com/books/bayesian-search-optimal-stopping-exploration/
SHA-256 · bayesian-search-optimal-stopping-exploration_bilingual_v1.0.0.pdf2624a5acfa910e3cc3cddd99ad7812335d4ce36aa197fa743acfe12cd3f6e1d0
SHA-256 · bayesian-search-optimal-stopping-exploration_en_v1.0.0.pdf27f83823d1fa701c986daea2c9cb0595e817088427f232edbbc1771e969eb5c1
SHA-256 · bayesian-search-optimal-stopping-exploration_zh_v1.0.0.pdf3a8f24c372bd8bd587f982057bdabf24be785dcbf1ea86164dbc58f74f8703b7
Licence · 许可Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0). https://creativecommons.org/licenses/by-nc-nd/4.0/

Disclosure. The author uses the works of 5D Library, including this book, as the methodological basis of his community courses, the 5D Ascension Cards (Dimensional Shift Card Decks) and the Yafu AI knowledge base, and therefore has a commercial interest in the subject of this book. The author also founded and runs Asia Influencer X and Yafu Global, whose business involves screening influencers and talent and testing new lanes and new projects, and therefore has a direct commercial interest in the subject of this book. The book promotes no course, card deck or service, is not investment, start-up or career advice, and uses no client data from these businesses; the Chapter 8 trial is designed to be run and analysed by researchers independent of the author's business and does not recruit from the author's community, students or customers, including the clients and students of these businesses; all the code and the study protocol are published openly for anyone to check. All models in this book are offered solely for discussion of mathematics and statistical decision theory, and the book recommends no specific project, lane or candidate.