Autonomous Nightly Funnel Micro-Surgery, Bandit Traffic Allocation and Zero-Touch Conversion Self-Improvement: chapter answers
The popular belief holds that a website no longer needs anyone watching it: in the small hours a program finds the leakiest step of the funnel and generates candidate changes, and by day Thompson sampling tilts traffic towards the best version and deletes the weak ones, so that conversion climbs by itself. This book examines the four hurdles such a machine must clear and makes four corrections to its blueprint. First, a bandit is designed to pay little regret, not to produce trustworthy effect sizes: the naive means of adaptive data come out 7.0% too low and must be corrected by inverse proba…
DOI pending · monograph page goes live after publication · 5DL-B-405 · 中文版
- Chapter 1 · The Funnel Check-up: Six-Step Beta Posteriors, Nightly Change Priorities and a Compliant Candidate Pool
Night B uses one night's small counts: 1,650 people enter, followed by 1,010, 445, 190, 112 and 77. Now the sample step has P(top) of only 0.714 and the first screen 0.286. The sample step's priority interval runs from… - Chapter 2 · Thompson Sampling and Regret: from Two Small Samples in 1933 to the Even Split
Chapter 2 compares the even split, explore-then-commit, UCB1 and Thompson sampling; - Chapter 3 · The Reward Function: Net Contribution per Visitor, Not the Conversion Rate
With the pay rate as reward, the realised pay rate is 0.0550 and net contribution per visitor 1.7140; the discount version takes 0.623 of the traffic and the trust version only 0.124. With net contribution as reward the… - Chapter 4 · Non-stationary Traffic: Weekday Effects, Discounted Memory and Sliding Windows
A per-day discount of 0.9 gives B 0.345 in the first week, 0.707 in the second, and settles at 0.843 and 0.847 in the third and fourth, for a total regret of 105.5 orders. A discount of 0.7 reacts faster in the first we… - Chapter 5 · Elimination Rules: Minimum Exposure, Posterior Win Probability and Time-uniform Confidence Sequences
Take the four lines at test level 0.05 first. The false-deletion rate lies between 0.003 and 0.006, and the deletion rate of the good version is always 0.000. The share of weak versions deleted by night 28 lies between… - Chapter 6 · Trustworthy Data: Negative Bias in Adaptive Data, Sample Ratio Mismatch and Guardrail Rollback
Chapter 6 the negative bias of adaptive data, sample ratio mismatch (a logging fault alone can manufacture a lift of 3.1%) and guardrail rollback. - Chapter 7 · Classical Sources East and West: "At the Limit, Change" in the Great Commentary, the Sunzi on Weak and Strong Points, and Random Allocation in Peirce and Jastrow
Chapter 7 examines the Great Commentary on the Book of Changes on change at the limit, the Sunzi on weak and strong points, and the random allocation of Peirce and Jastrow in 1885, and gives the distillation spec for th… - Chapter 8 · The Nightly Funnel Ledger: Field Tools and Checker Specifications, a Pre-registered Test and the Conditions of Refutation
Chapter 8 gives the nightly funnel ledger, the calculator, checkers grouped by the four kits and a pre-registered test: under the base prior the median cumulative gain in net contribution over half a year is about +0.08…
Illustrative model calculation, not a live quote