What does Creator Certification: Signals and Grades conclude, in numbers, about “The Early Pass Rate: Where to Set the Bar When Scores Are Noisy”?
Answer: When scores carry error, a lower pass rate gives higher precision, and on this point the popular belief is right; but recall, the number of people available and the information carried all fall. Program p21 shows that information is greatest when the pass rate is close to the base rate of strong creators, peaking at 0.35 for an error standard deviation of 0.5. Program p22 shows that once the decisions of brands are included, the best pass rate is 0.22, and the range within nine tenths of the best runs only from 0.18 to 0.23. Program p23 shows that after 10 early re-audits with no failure, the upper bound on the false-pass rate is still 0.238, so the platform cannot yet claim any precision.
Scope of the numbers: Its numbers of creators and brands, scoring errors, costs, premiums and audit parameters are illustrative, and all calculations are generated by its programs. It is general business guidance, not individual, legal, tax or investment advice.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.18, 0.22, 0.23
| pass | precision | brands filtering | demand | holders | filled | uplift |
|---|---|---|---|---|---|---|
| 0.05 | 0.995 | 200.0 | 600.0 | 100 | 100.0 | 184.5 |
| 0.10 | 0.975 | 197.1 | 591.4 | 200 | 200.0 | 313.9 |
| 0.15 | 0.939 | 178.6 | 535.8 | 300 | 300.0 | 417.1 |
| 0.20 | 0.894 | 154.5 | 463.5 | 400 | 400.0 | 500.8 |
| 0.25 | 0.841 | 127.0 | 380.9 | 500 | 380.9 | 433.0 |
| 0.30 | 0.786 | 97.8 | 293.4 | 600 | 293.4 | 304.2 |
| 0.40 | 0.676 | 39.9 | 119.8 | 800 | 119.8 | 103.5 |
| 0.50 | 0.577 | 0.0 | 0.0 | 1000 | 0.0 | 0.0 |
| 0.60 | 0.494 | 0.0 | 0.0 | 1200 | 0.0 | 0.0 |
| best pass rate 0.22: uplift 519.2, precision 0.873, 440 holders | ||||||
| within 90% of best: pass rates 0.18 to 0.23 | ||||||
| at 0.05 holders are too few; at 0.50 brands stop filtering |
Show the full program
# Chapter 2. p22: neither the lowest pass rate nor the most holders is best.
# 2,000 applicants a quarter; 200 brands, each filling 3 campaign slots.
# A brand filters on the certificate only if its precision P(strong | pass)
# reaches the brand's own requirement, spread uniformly on [0.60, 0.98].
# Slots that filtering brands fill from holders yield a quality uplift of
# E[q | pass] standard units each. Holders are scarce when r * 2,000 is
# below the filtering demand. Quality model as in p21 (sigma 0.5, base
# rate 0.30). All parameters are illustrative.
import math
from statistics import NormalDist
ND = NormalDist()
BASE, SIGMA = 0.30, 0.5
C_STRONG = ND.inv_cdf(1 - BASE)
GRID = [-6.0 + i * 0.002 for i in range(6001)]
APPLICANTS, BRANDS, SLOTS = 2000, 200, 3
REQ_LO, REQ_HI = 0.60, 0.98
def stats(r):
t = ND.inv_cdf(1 - r) * math.sqrt(1 + SIGMA * SIGMA)
joint = [(x, ND.pdf(x) * 0.002 * (1 - ND.cdf((t - x) / SIGMA)))
for x in GRID]
p_pass = math.fsum(v for _, v in joint)
p_ps = math.fsum(v for x, v in joint if x > C_STRONG)
return p_ps / p_pass, math.fsum(x * v for x, v in joint) / p_pass
def value(r):
prec, eq = stats(r)
share = min(1.0, max(0.0, (prec - REQ_LO) / (REQ_HI - REQ_LO)))
demand = BRANDS * share * SLOTS
holders = r * APPLICANTS
filled = min(demand, holders)
return prec, share, demand, holders, filled, filled * eq
print("pass precision brands filtering demand holders filled uplift")
rows = []
for k in range(2, 61):
r = k / 100
rows.append((r,) + value(r))
for row in rows:
r = row[0]
if round(r * 100) in (5, 10, 15, 20, 25, 30, 40, 50, 60):
print("%.2f %.3f %5.1f %5.1f %5.0f %5.1f %6.1f" % (
r, row[1], row[2] * BRANDS, row[3], row[4], row[5], row[6]))
best = max(rows, key=lambda x: x[6])
band = [x[0] for x in rows if x[6] >= 0.9 * best[6]]
print("best pass rate %.2f: uplift %.1f, precision %.3f, %d holders" % (
best[0], best[6], best[1], round(best[4])))
print("within 90%% of best: pass rates %.2f to %.2f" % (band[0], band[-1]))
print("at 0.05 holders are too few; at 0.50 brands stop filtering")Single source and citation
Fan, Y. (2026). Bootstrapping Industry Accreditation Standards, Spence Signalling Tiers and Inbound Creator Certification Magnetics (Version 1.0.0), Chapter 2: The Early Pass Rate: Where to Set the Bar When Scores Are Noisy. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-358 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026358,
author = {Fan, Yafu},
title = {Bootstrapping Industry Accreditation Standards, Spence Signalling Tiers and Inbound Creator Certification Magnetics},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Signalling Equilibrium: When a Certificate Separates Creators and When It Pools Them
- Chapter 3 · Grades: How Many Levels, Where to Cut, and Why Grade Inflation Is Sweet Before It Turns Bitter
- Chapter 4 · The Holder Premium: How Many Inquiries a Badge Brings, and How to Measure It Honestly
- Chapter 5 · Filtering by Brands and the Opportunity Cost of Not Certifying: When It Is Real and When It Is Exaggerated
- Chapter 6 · The Badge Display Cascade: Thresholds, Informational Cascades, Audits and Revocation
- Chapter 7 · Classical Sources East and West: "Inscribe the Maker's Name to Examine Sincerity" in the Book of Rites, and English Hallmarking
- Chapter 8 · System Architecture and Operating Specification: Application Flow, Public Registry, Weekly Ledger and Pre-Registered Tests