5D Library › Order-Bump Friction Elasticity, One-Click Upsell Kinetics and Cross-Domain Recommendation Matrices · 中文
Chapter 8. Incremental Revenue, Guardrails and a Pre-Registered Holdout Experiment
Attributed revenue is not incremental revenue
Calculation
# Chapter 8. p14: How much cross-sell revenue is truly caused?
# Some readers routed from site i to site j would have found j anyway.
# With a random holdout (no recommendation shown), incremental revenue =
# (rate with recommendation - rate in holdout) x value. Attributed
# revenue counts every purchase after a click. Illustrative rates.
def report(shown_rate, holdout_rate, click_share, value=98.0, n=10000):
attributed = n * shown_rate * click_share * value
incremental = n * (shown_rate - holdout_rate) * value
return attributed, incremental
if __name__ == "__main__":
print(" buy rate shown holdout attributed RM incremental RM share")
for s, h in ((0.020, 0.005), (0.020, 0.012), (0.020, 0.018)):
a, i = report(s, h, 0.9)
print(" {:14.3f} {:7.3f} {:13,.0f} {:14,.0f} {:.3f}".format(
s, h, a, i, i / a))
# buy rate shown holdout attributed RM incremental RM share
# 0.020 0.005 17,640 14,700 0.833
# 0.020 0.012 17,640 7,840 0.444
# 0.020 0.018 17,640 1,960 0.111A reader who is recommended a site, clicks and buys did not necessarily buy because of the recommendation. Suppose readers shown the recommendation buy at 0.020, and attributed revenue credits every purchase after a click to the recommendation. If a randomly held-out group shown no recommendation buys at 0.005, incremental revenue is 0.833 of attributed revenue; if the holdout buys at 0.012, only 0.444; at 0.018, only 0.111. Without a holdout there is no way to tell purchases the recommendation “caused” from purchases readers would have made anyway (Kohavi et al. 2020).
Guardrails
Calculation
# Chapter 8. p15: Base rates a bump and upsell programme must watch.
# For every 1,000 orders: bump takes, upsell acceptances, and the refunds
# and chargebacks each step adds (rates from p02, p03, p04; chargeback
# fee RM 60). A programme is healthy only if the added refunds cost far
# less than the added revenue.
if __name__ == "__main__":
orders = 1000
bump_rev = orders * 0.501 * 44
up_rev = orders * 0.13 * 300
bump_ref = orders * 0.501 * 0.04 * 44
up_ref = orders * 0.13 * (0.12 * 300 + 0.12 * 0.5 * 98)
cb = orders * 0.13 * 0.01 * (300 + 60)
print(" added revenue: bumps RM {:,.0f}, upsells RM {:,.0f}".format(
bump_rev, up_rev))
print(" refunds: bumps RM {:,.0f}, upsells RM {:,.0f};"
" chargebacks RM {:,.0f}".format(bump_ref, up_ref, cb))
net = bump_rev + up_rev - bump_ref - up_ref - cb
print(" net added RM {:,.0f}; refunds and chargebacks take {:.1%}"
.format(net, 1 - net / (bump_rev + up_rev)))
# added revenue: bumps RM 22,044, upsells RM 39,000
# refunds: bumps RM 882, upsells RM 5,444; chargebacks RM 468
# net added RM 54,250; refunds and chargebacks take 11.1%Per 1,000 orders, bumps bring RM 22,044 and upsells RM 39,000; refunds are RM 882 and RM 5,444, chargebacks RM 468. The net addition is RM 54,250, with refunds and chargebacks taking 11.1%. This share should be reported alongside bump and upsell revenue; if it rises, the bumps or upsells are starting to sell to people who should not buy.
Note
A four-step check. First, bumps are unticked by default with visible prices. Second, upsells credit the amount paid in full, and between the first second and day 7 leave one chance to change one’s mind. Third, history-based cross-site recommendation needs prior consent, with content-based recommendation the default. Fourth, the effects of recommendations and bumps are measured as increments against a holdout, with the refund rate as a guardrail.
A pre-registered holdout experiment
Calculation
# Chapter 8. p16: A pre-registered holdout test of cross-fleet routing.
# Readers are randomised: 90% see the two routed links, 10% (holdout)
# see none. Primary outcome: purchase on the target site within 30 days.
# Detect 2.0% vs 1.2% (two-sided 0.05, power 0.80) with unequal arms.
from math import sqrt, ceil
from statistics import NormalDist
N = NormalDist()
def n_unequal(p1, p2, ratio, alpha=0.05, power=0.80):
# n2 = holdout size, n1 = ratio x n2
za, zb = N.inv_cdf(1 - alpha / 2), N.inv_cdf(power)
var = p1 * (1 - p1) / ratio + p2 * (1 - p2)
return ceil((za + zb) ** 2 * var / (p1 - p2) ** 2)
if __name__ == "__main__":
print(" outcome: purchase within 30 days; 2.0% vs 1.2%")
print(" test: two-sided 0.05, power 0.80")
for share in (0.5, 0.2, 0.1):
ratio = (1 - share) / share
n2 = n_unequal(0.020, 0.012, ratio)
print(" holdout {:.0%}: holdout {:,} readers, total {:,}".format(
share, n2, ceil(n2 * (1 + ratio))))
# outcome: purchase within 30 days; 2.0% vs 1.2%
# test: two-sided 0.05, power 0.80
# holdout 50%: holdout 3,858 readers, total 7,716
# holdout 20%: holdout 2,055 readers, total 10,275
# holdout 10%: holdout 1,722 readers, total 17,220Readers are randomised into two groups: most see the two routed recommendations, a small holdout sees none. The primary outcome is a purchase on the target site within 30 days, testing 2.0% against 1.2% (two-sided 0.05, power 0.80). With a half holdout, the holdout needs 3,858 readers and the total 7,716; with a fifth, 2,055 and 10,275; with a tenth, 1,722 and 17,220. The smaller the holdout, the less it touches day-to-day revenue, but the more readers are needed in all.
Hypothesis · not established
Pre-registered predictions (not yet tested): the incremental revenue of cross-fleet recommendations is no more than half their attributed revenue; refunds and chargebacks take less than 15% of bump and upsell revenue. What would show them wrong: incremental revenue significantly above half of attributed revenue, or refunds and chargebacks above 15%.
Summary of the book
The advantage of the checkout bump comes from zero friction, and the best price ratio is set by the valuation distribution; large lifts in order value need a bump, a second bump and an upsell together. The main peak of the upsell is the first second after payment and the second peak day 7; counting refunds, the first second still wins. Mutual-information routing is unstable in the synthetic model and should rotate within groups. Using personal data across sites needs consent. The true value of recommendations must be measured against a holdout.
References
Fan, Y. (2026). Order-Bump Friction Elasticity, One-Click Upsell Kinetics and Cross-Domain Recommendation Matrices: Zero-Reentry Impulse Attachment, Conditional Mutual Information and Cart-Value Multipliers [结账顺手加购弹性、一键升单动力学与跨领域推荐矩阵] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090207 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).