What does Star Ratings, Self-Selection and Bayesian Ranking conclude, in numbers, about “Sorting Is Soliciting: the Shelf Feedback Loop, the Cost of Exploring and the First Five Reviews”?
Answer: A sort key is also a policy for collecting reviews. On the illustrative shelf, the beta posterior mean loses 85.8 orders and tests 6.12 products; the Wilson bound, the most cautious, tests only 3.00 and loses 118.8; Thompson sampling tests 18.38 and loses 131.6 in the short run. The first 5 reviews raise the weight of a product's own data to 0.333, while reviews 6 to 50 add only 0.500 more, so effort to solicit reviews should go first to products with fewer than 5. Do ratings, like rankings, drift as a whole because raters look at one another? Chapter 6 takes up rating inflation.
Scope of the numbers: Chapter 1 explains the J-shaped star distribution with two filters, a purchasing bias and an under-reporting bias; with illustrative parameters the purchasing bias lifts the mean by +0.532 stars. Marked boxes separate published evidence, runnable Python calculations (21 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 118.8, 131.6, 18.38, 3.00, 6.12, 85.8
| 6000 shoppers, 40 listings, mean of 8 shelves (illustrative) | |||
| policy | bought | regret listings>=7 | |
| raw | 1457.8 | 101.5 | 13.25 |
| wilson | 1440.4 | 118.8 | 3.00 |
| bayes | 1473.4 | 85.8 | 6.12 |
| thompson | 1427.6 | 131.6 | 18.38 |
| oracle | 1559.2 | 0.0 | - |
Show the full program
# Chapter 5. p51: the sort key is also a review-collection policy.
# Shelf of 40 listings, true positive share p ~ Beta(8, 2), mean 0.8
# (illustrative);
# each listing starts with 2 seeded reviews drawn from its own p.
# Each round one shopper sees the top 3 slots with view probabilities
# 1.0, 0.6, 0.35, buys a viewed item with prob 0.15 p_j (the first
# purchase ends the visit); a buyer reviews with prob 0.3, positive with
# prob p_j.
# Policies: raw share, Wilson LB, Beta posterior mean, Thompson sampling
# (rank by one draw from each Beta posterior). 6000 rounds, 8 shelves.
# Regret = purchases lost against always showing the true best three.
import math
import random
Z = 1.959964
A0, B0 = 8.0, 2.0
VIEW = (1.0, 0.6, 0.35)
ROUNDS, SHELVES, N_ITEMS = 6000, 8, 40
BUY, REVIEW = 0.15, 0.3
def wilson_lb(k, n):
p = k / n
a = p + Z * Z / (2 * n)
b = Z * math.sqrt(p * (1 - p) / n + Z * Z / (4 * n * n))
return (a - b) / (1 + Z * Z / n)
def oracle_rate(ps):
top = sorted(ps, reverse=True)[:3]
miss, rate = 1.0, 0.0
for v, p in zip(VIEW, top):
rate += miss * v * BUY * p
miss *= 1 - v * BUY * p
return rate
def run(rng, ps, policy):
k = [0] * N_ITEMS
n = [0] * N_ITEMS
for j in range(N_ITEMS):
for _ in range(2):
n[j] += 1
k[j] += 1 if rng.random() < ps[j] else 0
bought = 0
for _ in range(ROUNDS):
if policy == "thompson":
key = [rng.betavariate(k[j] + A0, n[j] - k[j] + B0)
for j in range(N_ITEMS)]
elif policy == "raw":
key = [k[j] / n[j] for j in range(N_ITEMS)]
elif policy == "wilson":
key = [wilson_lb(k[j], n[j]) for j in range(N_ITEMS)]
else:
key = [(k[j] + A0) / (n[j] + A0 + B0) for j in range(N_ITEMS)]
order = sorted(range(N_ITEMS), key=lambda j: (-key[j], j))[:3]
for v, j in zip(VIEW, order):
if rng.random() < v and rng.random() < BUY * ps[j]:
bought += 1
if rng.random() < REVIEW:
n[j] += 1
k[j] += 1 if rng.random() < ps[j] else 0
break
reviewed = len([j for j in range(N_ITEMS) if n[j] >= 7])
return bought, reviewed
rng = random.Random(20261112)
shelves = [[rng.betavariate(A0, B0) for _ in range(N_ITEMS)]
for _ in range(SHELVES)]
best = math.fsum(oracle_rate(ps) * ROUNDS for ps in shelves) / SHELVES
print("6000 shoppers, 40 listings, mean of 8 shelves (illustrative)")
print("%-10s %9s %9s %11s" % ("policy", "bought", "regret",
"listings>=7"))
for pol in ("raw", "wilson", "bayes", "thompson"):
b, r = [], []
for ps in shelves:
x, y = run(rng, ps, pol)
b.append(x)
r.append(y)
mb = math.fsum(b) / SHELVES
print("%-10s %9.1f %9.1f %11.2f" % (pol, mb, best - mb,
math.fsum(r) / SHELVES))
print("%-10s %9.1f %9.1f %11s" % ("oracle", best, 0.0, "-"))Single source and citation
Fan, Y. (2026). J-Shaped Self-Selection in Star Ratings, Bayesian-Average Ranking and Rating Inflation from the Nine-Rank System to Online Marketplaces (Version 1.0.0), Chapter 5: Sorting Is Soliciting: the Shelf Feedback Loop, the Cost of Exploring and the First Five Reviews. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-212 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026212,
author = {Fan, Yafu},
title = {J-Shaped Self-Selection in Star Ratings, Bayesian-Average Ranking and Rating Inflation from the Nine-Rank System to Online Marketplaces},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Where the J Comes From: a U-Shaped Urge to Post and Two Self-Selection Biases
- Chapter 2 · Five Stars Do Not Sell Best: the Spiegel Curve at Source and a Small-Sample Reading
- Chapter 3 · The Wilson Lower Bound: Why Three Five-Star Reviews Rank Below Two Hundred at 4.6
- Chapter 4 · The Dirichlet-Prior Bayesian Average: Pseudo-Reviews Borrowed from the Shop
- Chapter 6 · Rating Inflation: Norm Chasing, Information Decay and the Friction of Asking
- Chapter 7 · Classic Texts East and West: Liu Yi on the Nine-Rank System and Galton's "Vox Populi"
- Chapter 8 · Closing the Loop: Shop Sorting, the Timing of Review Requests and a Falsification Plan