What does Star Ratings, Self-Selection and Bayesian Ranking conclude, in numbers, about “Sorting Is Soliciting: the Shelf Feedback Loop, the Cost of Exploring and the First Five Reviews”?

Answer: A sort key is also a policy for collecting reviews. On the illustrative shelf, the beta posterior mean loses 85.8 orders and tests 6.12 products; the Wilson bound, the most cautious, tests only 3.00 and loses 118.8; Thompson sampling tests 18.38 and loses 131.6 in the short run. The first 5 reviews raise the weight of a product's own data to 0.333, while reviews 6 to 50 add only 0.500 more, so effort to solicit reviews should go first to products with fewer than 5. Do ratings, like rankings, drift as a whole because raters look at one another? Chapter 6 takes up rating inflation.

Scope of the numbers: Chapter 1 explains the J-shaped star distribution with two filters, a purchasing bias and an under-reporting bias; with illustrative parameters the purchasing bias lifts the mean by +0.532 stars. Marked boxes separate published evidence, runnable Python calculations (21 programs), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 118.8, 131.6, 18.38, 3.00, 6.12, 85.8

6000 shoppers, 40 listings, mean of 8 shelves (illustrative)
policyboughtregret listings>=7
raw1457.8101.513.25
wilson1440.4118.83.00
bayes1473.485.86.12
thompson1427.6131.618.38
oracle1559.20.0-
Show the full program
# Chapter 5. p51: the sort key is also a review-collection policy.
# Shelf of 40 listings, true positive share p ~ Beta(8, 2), mean 0.8
# (illustrative);
# each listing starts with 2 seeded reviews drawn from its own p.
# Each round one shopper sees the top 3 slots with view probabilities
# 1.0, 0.6, 0.35, buys a viewed item with prob 0.15 p_j (the first
# purchase ends the visit); a buyer reviews with prob 0.3, positive with
# prob p_j.
# Policies: raw share, Wilson LB, Beta posterior mean, Thompson sampling
# (rank by one draw from each Beta posterior). 6000 rounds, 8 shelves.
# Regret = purchases lost against always showing the true best three.
import math
import random

Z = 1.959964
A0, B0 = 8.0, 2.0
VIEW = (1.0, 0.6, 0.35)
ROUNDS, SHELVES, N_ITEMS = 6000, 8, 40
BUY, REVIEW = 0.15, 0.3


def wilson_lb(k, n):
    p = k / n
    a = p + Z * Z / (2 * n)
    b = Z * math.sqrt(p * (1 - p) / n + Z * Z / (4 * n * n))
    return (a - b) / (1 + Z * Z / n)


def oracle_rate(ps):
    top = sorted(ps, reverse=True)[:3]
    miss, rate = 1.0, 0.0
    for v, p in zip(VIEW, top):
        rate += miss * v * BUY * p
        miss *= 1 - v * BUY * p
    return rate


def run(rng, ps, policy):
    k = [0] * N_ITEMS
    n = [0] * N_ITEMS
    for j in range(N_ITEMS):
        for _ in range(2):
            n[j] += 1
            k[j] += 1 if rng.random() < ps[j] else 0
    bought = 0
    for _ in range(ROUNDS):
        if policy == "thompson":
            key = [rng.betavariate(k[j] + A0, n[j] - k[j] + B0)
                   for j in range(N_ITEMS)]
        elif policy == "raw":
            key = [k[j] / n[j] for j in range(N_ITEMS)]
        elif policy == "wilson":
            key = [wilson_lb(k[j], n[j]) for j in range(N_ITEMS)]
        else:
            key = [(k[j] + A0) / (n[j] + A0 + B0) for j in range(N_ITEMS)]
        order = sorted(range(N_ITEMS), key=lambda j: (-key[j], j))[:3]
        for v, j in zip(VIEW, order):
            if rng.random() < v and rng.random() < BUY * ps[j]:
                bought += 1
                if rng.random() < REVIEW:
                    n[j] += 1
                    k[j] += 1 if rng.random() < ps[j] else 0
                break
    reviewed = len([j for j in range(N_ITEMS) if n[j] >= 7])
    return bought, reviewed


rng = random.Random(20261112)
shelves = [[rng.betavariate(A0, B0) for _ in range(N_ITEMS)]
           for _ in range(SHELVES)]
best = math.fsum(oracle_rate(ps) * ROUNDS for ps in shelves) / SHELVES
print("6000 shoppers, 40 listings, mean of 8 shelves (illustrative)")
print("%-10s %9s %9s %11s" % ("policy", "bought", "regret",
                               "listings>=7"))
for pol in ("raw", "wilson", "bayes", "thompson"):
    b, r = [], []
    for ps in shelves:
        x, y = run(rng, ps, pol)
        b.append(x)
        r.append(y)
    mb = math.fsum(b) / SHELVES
    print("%-10s %9.1f %9.1f %11.2f" % (pol, mb, best - mb,
                                       math.fsum(r) / SHELVES))
print("%-10s %9.1f %9.1f %11s" % ("oracle", best, 0.0, "-"))

Single source and citation

Fan, Y. (2026). J-Shaped Self-Selection in Star Ratings, Bayesian-Average Ranking and Rating Inflation from the Nine-Rank System to Online Marketplaces (Version 1.0.0), Chapter 5: Sorting Is Soliciting: the Shelf Feedback Loop, the Cost of Exploring and the First Five Reviews. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-212 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026212,
  author    = {Fan, Yafu},
  title     = {J-Shaped Self-Selection in Star Ratings, Bayesian-Average Ranking and Rating Inflation from the Nine-Rank System to Online Marketplaces},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book