5D Library › Diagnostic-to-Pedagogy Conversion Funnels, Mastery-Gated Curriculum Architecture and Certification Network Economics · 中文
Chapter 6. Two-Sided Networks, Grading Noise and Teaching to the Test
The client side: a two-sided market
Chapter 5 assumed that every licensee always holds 100 sessions a year, however many licensees there are. That holds when licensees are few and fails when they are many: clients are a finite pool, and the more licensees, the fewer clients each gets. A network of licensed practitioners is a two-sided market: clients care about practitioners’ quality and how many there are to choose from; practitioners care about how many clients they get (Rochet and Tirole 2003; Armstrong 2006).
The program lets the total sessions clients book each year rise with quality and with the number of practitioners (); practitioners keep 90% of session revenue; a person who has passed takes up the licence only if session income less the licence fee of 600 exceeds her outside option (uniform between 0 and 8,000). The number of active practitioners solves a fixed-point equation, found by bisection.
Calculation
# Chapter 6. [p11] Two sides: certified practitioners and their clients.
# For each pass mark T the certified pool of p09 has size N_c(T) and mean
# mastery Q(T). Clients book S = 50,000 exp(4 (Q - 0.75)) (N / 500)^0.3
# sessions a year: they want quality and some choice. Price per session
# 80 exp(4 (Q - 0.75)); the practitioner keeps 90%. A certified person
# takes up the licence (fee 600) if income S p 0.9 / N - 600 exceeds an
# outside option drawn uniform on 0..8,000, so the active number N
# solves N = N_c F(S p 0.9 / N - 600), found by bisection. Firm profit
# = 600 N + 0.1 S p. The one-sided benchmark (p09) assumes 100 sessions
# per licensee whatever their number.
import math
K = 6.0
def total_pmf(m):
a, b = K * m, K * (1 - m)
lb = lambda x, y: math.lgamma(x) + math.lgamma(y) - math.lgamma(x + y)
one = [math.exp(math.lgamma(5) - math.lgamma(j + 1) - math.lgamma(5 - j)
+ lb(j + a, 4 - j + b) - lb(a, b)) for j in range(5)]
tot = [1.0]
for _ in range(5):
nxt = [0.0] * (len(tot) + 4)
for i, x in enumerate(tot):
for j, y in enumerate(one):
nxt[i + j] += x * y
tot = nxt
return tot
def pool(T, grid=400):
n = q = 0.0
for i in range(grid):
th = (i + 0.5) / grid
w = 168 * th ** 5 * (1 - th) ** 2 / grid
m = min(0.97, th + 0.08)
pp = sum(total_pmf(m)[T:])
n += w * pp
q += w * pp * m
return 1000 * n, q / n
def equilibrium(nc, Q):
price = 80 * math.exp(4 * (Q - 0.75))
def share(N):
S = 50000 * math.exp(4 * (Q - 0.75)) * (N / 500) ** 0.3
inc = S * price * 0.9 / N - 600
return min(1.0, max(0.0, inc / 8000)), S
lo, hi = 1e-6, nc
for _ in range(80):
mid = (lo + hi) / 2
if nc * share(mid)[0] > mid:
lo = mid
else:
hi = mid
S = share(lo)[1]
return lo, S, price, 600 * lo + 0.1 * S * price
if __name__ == "__main__":
print(" T certified active sessions two-sided one-sided")
best2 = best1 = None
prof2 = {}
for T in range(10, 21):
nc, Q = pool(T)
N, S, p, prof = equilibrium(nc, Q)
one = nc * (600 + 10 * p)
prof2[T] = prof
best2 = max(best2 or (prof, T), (prof, T))
best1 = max(best1 or (one, T), (one, T))
print(" {:2d} {:10.1f} {:7.1f} {:10.0f} {:10.0f} {:10.0f}".format(
T, nc, N, S, prof, one))
print(" best T: two-sided {}, one-sided {}".format(best2[1], best1[1]))
print(" two-sided profit at T = 16 / at best T: {:.4f}".format(
prof2[16] / best2[0]))
print("\nReading: once clients are a finite pool shared among")
print("licensees, a lax mark certifies people who then find too")
print("few clients to take the licence; the cross-side effect moves")
print("the best mark from 10 to 18, and the curve is flat enough")
print("that 16/20 gives up less than one per cent.")
# T certified active sessions two-sided one-sided
# 10 909.4 687.1 59060 919613 1326784
# 11 868.0 697.9 61374 964040 1291977
# 12 815.2 708.0 64183 1018617 1243348
# 13 750.5 715.9 67508 1084272 1178226
# 14 674.0 674.0 69971 1120621 1094334
# 15 586.7 586.7 71314 1127620 990152
# 16 490.7 490.7 72298 1135464 865282
# 17 389.0 389.0 72607 1142861 720647
# 18 285.5 285.5 71715 1144817 558862
# 19 184.5 184.5 68583 1125667 383736
# 20 89.4 89.4 60407 1032295 198401
# best T: two-sided 18, one-sided 10
# two-sided profit at T = 16 / at best T: 0.9918
#
# Reading: once clients are a finite pool shared among
# licensees, a lax mark certifies people who then find too
# few clients to take the licence; the cross-side effect moves
# the best mark from 10 to 18, and the curve is flat enough
# that 16/20 gives up less than one per cent.The result turns Chapter 5’s static conclusion around. With a mark of 10, 909.4 pass but only 687.1 take up the licence: the client pool is spread thin, and many who pass get too few clients to make the fee worthwhile. As the mark rises, fewer pass but each gets more clients and the quality premium rises; from 14 upward every passer takes the licence. The most profitable mark in the two-sided equilibrium is 18, and profit at 16 is 0.9918 of the optimum, less than one per cent short. The profit curve is very flat between 14 and 19.
This section is the result in the book most favourable to the author’s outline: once the client side is treated as a finite pool that is spread thin, 16/20 is close to optimal. But it rests on the elasticity of client demand with respect to the number of practitioners (0.3 here) and on the distribution of outside options, both of which need real data.
Noise in AI grading
The false passes and false fails of Chapter 3 come from the coarseness of five questions. AI grading adds a second layer of noise: the program lets the grader mark a question one point too high or too low with probability (half each way, kept between 0 and 4), and compares a “borderline review” policy, under which scripts graded 15 or 16 go to a person who marks them without error.
Calculation
# Chapter 6. [p12] A noisy AI grader and a human look at the borderline.
# Exam and population as in p05 (5 questions x 4 points, pass 16, point
# correlation k = 6, candidates Beta(6, 3), competent m >= 0.80). The
# grader mis-scores each question with probability e by one point, up
# or down with equal chance (clipped to 0..4). The joint law of the true
# and the graded total is convolved exactly over the 5 questions.
# Policy B (borderline review): a script graded 15 or 16 goes to a
# human who scores it without error. Reported: pass rate, false passes
# among passers, and the competent who fail.
import math
K = 6.0
def bb(m):
a, b = K * m, K * (1 - m)
lb = lambda x, y: math.lgamma(x) + math.lgamma(y) - math.lgamma(x + y)
return [math.exp(math.lgamma(5) - math.lgamma(j + 1) - math.lgamma(5 - j)
+ lb(j + a, 4 - j + b) - lb(a, b)) for j in range(5)]
def joint(m, e):
one = {}
for j, p in enumerate(bb(m)):
for d, w in ((0, 1 - e), (-1, e / 2), (1, e / 2)):
g = min(4, max(0, j + d))
one[(j, g)] = one.get((j, g), 0.0) + p * w
tot = {(0, 0): 1.0}
for _ in range(5):
nxt = {}
for (t, g), p in tot.items():
for (j, h), q in one.items():
key = (t + j, g + h)
nxt[key] = nxt.get(key, 0.0) + p * q
tot = nxt
return tot
def pass_prob(m, e, review):
s = 0.0
for (t, g), p in joint(m, e).items():
if review and g in (15, 16):
s += p * (t >= 16)
else:
s += p * (g >= 16)
return s
def metrics(e, review, n=400):
pas = fp = fn = comp = 0.0
for i in range(n):
m = (i + 0.5) / n
w = 168 * m ** 5 * (1 - m) ** 2 / n
pp = pass_prob(m, e, review)
pas += w * pp
if m >= 0.80:
comp += w
fn += w * (1 - pp)
else:
fp += w * pp
return pas, fp / pas, fn / comp
if __name__ == "__main__":
print(" e review pass rate false pass competent fail")
for e in (0.0, 0.1, 0.2, 0.3):
for rv in (False, True):
if e == 0.0 and rv:
continue
a, b, c = metrics(e, rv)
print(" {:.1f} {:7s} {:11.4f} {:11.4f} {:15.4f}".format(
e, "yes" if rv else "no", a, b, c))
print("\nReading: a question marked 4 can only be marked down, so")
print("grader noise mostly fails competent candidates and hardly")
print("changes the share of weak passers. A human review of scripts")
print("graded 15 or 16 recovers most of the damage. The exam's own")
print("coarseness (e = 0) remains the larger error.")
# e review pass rate false pass competent fail
# 0.0 no 0.3223 0.4930 0.1953
# 0.1 no 0.3145 0.4935 0.2156
# 0.1 yes 0.3211 0.4925 0.1977
# 0.2 no 0.3066 0.4944 0.2367
# 0.2 yes 0.3179 0.4914 0.2039
# 0.3 no 0.2986 0.4957 0.2585
# 0.3 yes 0.3134 0.4900 0.2131
#
# Reading: a question marked 4 can only be marked down, so
# grader noise mostly fails competent candidates and hardly
# changes the share of weak passers. A human review of scripts
# graded 15 or 16 recovers most of the damage. The exam's own
# coarseness (e = 0) remains the larger error.A surprising result: symmetric grading noise does not mainly let weak candidates through; it fails strong ones. The reason is the edge of the scale: a question with full marks of 4 can only be marked down, never up. At , the share of the competent who fail on the day rises from 0.1953 to 0.2585, while the false-pass share barely moves (0.4957). Borderline review brings false fails of the competent back to 0.2131.
So AI grading noise is not the main problem here; the coarseness of five questions is (at false passes are already 0.4930). Borderline review is a cheap patch with a clear effect.
Teaching to the test
Campbell observed that the more a quantitative indicator is used for social decisions, the more it is subject to corruption and the more it distorts the processes it was meant to monitor (Campbell 1979). Exams are no exception: once licence income depends on an exam, coaching aimed at the item bank appears, and the correlation between score and real skill falls.
The program describes this with a Gaussian model: exam performance correlates with real skill at , the pass mark always lets 32.23% through (the pass rate of Chapter 3), and the brand price follows the formula of Chapter 5.
Calculation
# Chapter 6. [p13] Teaching to the test: when the score stops tracking skill.
# Skill z and exam aptitude x are standard normal with correlation rho:
# x = rho z + sqrt(1 - rho^2) u. Mastery is m = 0.667 + 0.15 z (about
# the Beta(6, 3) of p05). The pass mark is set so that 32.23% pass, the
# p05 pass rate. Coaching aimed at the item bank lowers rho. For each
# rho the program reports (200,000 draws, fixed seed) the mean mastery
# of passers, the share of passers below 0.80, and the brand price of
# p09, 80 exp(4 (Q - 0.75)). A remedy: add k unseen transfer questions
# whose own correlation with skill is 0.8, averaged with the old score.
import math
import random
def sim(rho, rng, n=200000, transfer=0, rt=0.8):
xs, ms = [], []
for _ in range(n):
z = rng.gauss(0, 1)
x = rho * z + math.sqrt(1 - rho * rho) * rng.gauss(0, 1)
if transfer:
t = sum(rt * z + math.sqrt(1 - rt * rt) * rng.gauss(0, 1)
for _ in range(transfer)) / transfer
x = (x + transfer * t) / (1 + transfer)
xs.append(x)
ms.append(0.667 + 0.15 * z)
cut = sorted(xs)[int(n * (1 - 0.3223))]
pm = [m for x, m in zip(xs, ms) if x >= cut]
q = sum(pm) / len(pm)
low = sum(m < 0.80 for m in pm) / len(pm)
return q, low, 80 * math.exp(4 * (q - 0.75))
if __name__ == "__main__":
rng = random.Random(74013)
print(" rho transfer mean m below 0.80 brand price")
for rho in (0.9, 0.7, 0.5, 0.3):
q, low, p = sim(rho, rng)
print(" {:.1f} {:9d} {:8.4f} {:11.4f} {:12.2f}".format(
rho, 0, q, low, p))
for k in (1, 3):
q, low, p = sim(0.3, rng, transfer=k)
print(" {:.1f} {:9d} {:8.4f} {:11.4f} {:12.2f}".format(
0.3, k, q, low, p))
print("\nReading: as coaching decouples the score from skill, the")
print("pass pool drifts toward the population mean and the brand")
print("premium that justified the mark melts away. Unseen transfer")
print("questions - one corner shown, three to be found - restore")
print("most of it.")
# rho transfer mean m below 0.80 brand price
# 0.9 0 0.8177 0.4561 104.89
# 0.7 0 0.7839 0.5603 91.60
# 0.5 0 0.7509 0.6422 80.27
# 0.3 0 0.7176 0.7158 70.28
# 0.3 1 0.7841 0.5591 91.68
# 0.3 3 0.8150 0.4650 103.75
#
# Reading: as coaching decouples the score from skill, the
# pass pool drifts toward the population mean and the brand
# premium that justified the mark melts away. Unseen transfer
# questions - one corner shown, three to be found - restore
# most of it.As falls from 0.9 to 0.3, passers’ mean mastery falls from 0.8177 to 0.7176, the share of passers below 0.80 rises from 0.4561 to 0.7158, and the brand price falls from 104.89 to 70.28. The mark is still there and the pass rate has not changed, but what it filters out is no longer low skill.
The remedy is to add transfer questions the candidate has never seen: give one example and ask her to work out the other cases. Even when the old questions have been coached down to , one transfer question with correlation 0.8 brings mean mastery back to 0.7841, and three bring it back to 0.8150, close to the level at . This is exactly the meaning of “shown one corner, come back with the other three” in Chapter 7.
Hypothesis · not established
The author’s interpretation (not yet tested): in the author’s examinations, at least 3 of each subject’s 5 questions should be new cases from outside the bank (giving the method, not a similar worked example), which would keep the quality of the licensed group stable even after a coaching market appears. What would show this wrong: if real exam data showed that scores on bank questions and on new cases correlate equally with client ratings after licensing, transfer questions would measure skill no better and the requirement could be dropped.
Summary
Once the client side is treated as a finite pool, the best mark in the two-sided equilibrium is 18, and 16 is less than one per cent short. AI grading noise mainly hurts strong candidates, and borderline review repairs it. Teaching to the test empties the pass mark of its filtering power; unseen transfer questions bring it back.
References
Fan, Y. (2026). Diagnostic-to-Pedagogy Conversion Funnels, Mastery-Gated Curriculum Architecture and Certification Network Economics: Self-Cannibalization Bounds, Spaced-Retrieval Curves and Two-Sided Practitioner Licensing [诊学一体转化漏斗、掌握度门控课程与持牌人网络经济学] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090197 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).