5D Library › 诊学一体转化漏斗、掌握度门控课程与持牌人网络经济学 · English

第六章 双边网络、阅卷噪声与应试教学

客户一侧:双边市场

第五章假设每位持牌人每年总有 100 次服务,与持牌人数无关。这在持牌人很少时成立,持牌人一多就不成立:客户是一个有限的池子,持牌人越多,每人分到的越少。持牌执业者网络是一个双边市场:客户在乎执业者的质量与可选择的数量,执业者在乎能分到多少客户 (Rochet and Tirole 2003; Armstrong 2006)。

程序让客户每年预约的服务总数随质量与执业者数量上升(S=50000exp⁡(4(Q−0.75))(N/500)0.3S = 50000\exp(4(Q-0.75))(N/500)^{0.3}),执业者拿 90% 的服务收入;一位通过考试的人只有在服务收入减去 600 的牌照费后超过自己的外部机会(在 0 到 8,000 之间均匀分布)时才会领牌。活跃执业者数 NN 由不动点方程决定,用二分法求解。

推导

# Chapter 6. [p11] Two sides: certified practitioners and their clients.
# For each pass mark T the certified pool of p09 has size N_c(T) and mean
# mastery Q(T). Clients book S = 50,000 exp(4 (Q - 0.75)) (N / 500)^0.3
# sessions a year: they want quality and some choice. Price per session
# 80 exp(4 (Q - 0.75)); the practitioner keeps 90%. A certified person
# takes up the licence (fee 600) if income S p 0.9 / N - 600 exceeds an
# outside option drawn uniform on 0..8,000, so the active number N
# solves N = N_c F(S p 0.9 / N - 600), found by bisection. Firm profit
# = 600 N + 0.1 S p. The one-sided benchmark (p09) assumes 100 sessions
# per licensee whatever their number.
import math

K = 6.0

def total_pmf(m):
    a, b = K * m, K * (1 - m)
    lb = lambda x, y: math.lgamma(x) + math.lgamma(y) - math.lgamma(x + y)
    one = [math.exp(math.lgamma(5) - math.lgamma(j + 1) - math.lgamma(5 - j)
                    + lb(j + a, 4 - j + b) - lb(a, b)) for j in range(5)]
    tot = [1.0]
    for _ in range(5):
        nxt = [0.0] * (len(tot) + 4)
        for i, x in enumerate(tot):
            for j, y in enumerate(one):
                nxt[i + j] += x * y
        tot = nxt
    return tot

def pool(T, grid=400):
    n = q = 0.0
    for i in range(grid):
        th = (i + 0.5) / grid
        w = 168 * th ** 5 * (1 - th) ** 2 / grid
        m = min(0.97, th + 0.08)
        pp = sum(total_pmf(m)[T:])
        n += w * pp
        q += w * pp * m
    return 1000 * n, q / n

def equilibrium(nc, Q):
    price = 80 * math.exp(4 * (Q - 0.75))
    def share(N):
        S = 50000 * math.exp(4 * (Q - 0.75)) * (N / 500) ** 0.3
        inc = S * price * 0.9 / N - 600
        return min(1.0, max(0.0, inc / 8000)), S
    lo, hi = 1e-6, nc
    for _ in range(80):
        mid = (lo + hi) / 2
        if nc * share(mid)[0] > mid:
            lo = mid
        else:
            hi = mid
    S = share(lo)[1]
    return lo, S, price, 600 * lo + 0.1 * S * price

if __name__ == "__main__":
    print("    T  certified  active   sessions  two-sided  one-sided")
    best2 = best1 = None
    prof2 = {}
    for T in range(10, 21):
        nc, Q = pool(T)
        N, S, p, prof = equilibrium(nc, Q)
        one = nc * (600 + 10 * p)
        prof2[T] = prof
        best2 = max(best2 or (prof, T), (prof, T))
        best1 = max(best1 or (one, T), (one, T))
        print("   {:2d} {:10.1f} {:7.1f} {:10.0f} {:10.0f} {:10.0f}".format(
            T, nc, N, S, prof, one))
    print("   best T: two-sided {}, one-sided {}".format(best2[1], best1[1]))
    print("   two-sided profit at T = 16 / at best T: {:.4f}".format(
        prof2[16] / best2[0]))
    print("\nReading: once clients are a finite pool shared among")
    print("licensees, a lax mark certifies people who then find too")
    print("few clients to take the licence; the cross-side effect moves")
    print("the best mark from 10 to 18, and the curve is flat enough")
    print("that 16/20 gives up less than one per cent.")
#     T  certified  active   sessions  two-sided  one-sided
#    10      909.4   687.1      59060     919613    1326784
#    11      868.0   697.9      61374     964040    1291977
#    12      815.2   708.0      64183    1018617    1243348
#    13      750.5   715.9      67508    1084272    1178226
#    14      674.0   674.0      69971    1120621    1094334
#    15      586.7   586.7      71314    1127620     990152
#    16      490.7   490.7      72298    1135464     865282
#    17      389.0   389.0      72607    1142861     720647
#    18      285.5   285.5      71715    1144817     558862
#    19      184.5   184.5      68583    1125667     383736
#    20       89.4    89.4      60407    1032295     198401
#    best T: two-sided 18, one-sided 10
#    two-sided profit at T = 16 / at best T: 0.9918
#
# Reading: once clients are a finite pool shared among
# licensees, a lax mark certifies people who then find too
# few clients to take the licence; the cross-side effect moves
# the best mark from 10 to 18, and the curve is flat enough
# that 16/20 gives up less than one per cent.
单边静态利润与双边均衡利润随及格线变化(各自以最大值归一)。点线为 16 分。

结果把第五章的静态结论翻了过来。及格线为 10 时有 909.4 人通过,但只有 687.1 人领牌:客户池被摊薄,许多通过者分不到足够的客户,不值得付牌照费。及格线提高时,通过者减少,但每人分到的客户增多,质量溢价也上升;从 14 分起,所有通过者都领牌。双边均衡下利润最高的及格线是 18,而 16 分的利润是最优的 0.9918,只少不到百分之一。利润曲线在 14 到 19 之间非常平坦。

这一节是全书中对作者提纲最有利的结果:一旦把客户一侧当作有限的、会被摊薄的池子,16/20 就接近最优。但它依赖于客户需求对执业者数量的弹性(这里是 0.3)与外部机会的分布,这两者都需要真实数据。

AI 阅卷的噪声

第三章的误放与误拒来自 5 道题本身的粗糙。AI 阅卷还会引入第二层噪声:程序让阅卷器以概率 ee 把一道题的分数判高或判低一分(各一半,限制在 0 到 4 之间),并比较「边界复核」方案:被判为 15 或 16 分的试卷交给人工无误差复核。

推导

# Chapter 6. [p12] A noisy AI grader and a human look at the borderline.
# Exam and population as in p05 (5 questions x 4 points, pass 16, point
# correlation k = 6, candidates Beta(6, 3), competent m >= 0.80). The
# grader mis-scores each question with probability e by one point, up
# or down with equal chance (clipped to 0..4). The joint law of the true
# and the graded total is convolved exactly over the 5 questions.
# Policy B (borderline review): a script graded 15 or 16 goes to a
# human who scores it without error. Reported: pass rate, false passes
# among passers, and the competent who fail.
import math

K = 6.0

def bb(m):
    a, b = K * m, K * (1 - m)
    lb = lambda x, y: math.lgamma(x) + math.lgamma(y) - math.lgamma(x + y)
    return [math.exp(math.lgamma(5) - math.lgamma(j + 1) - math.lgamma(5 - j)
                     + lb(j + a, 4 - j + b) - lb(a, b)) for j in range(5)]

def joint(m, e):
    one = {}
    for j, p in enumerate(bb(m)):
        for d, w in ((0, 1 - e), (-1, e / 2), (1, e / 2)):
            g = min(4, max(0, j + d))
            one[(j, g)] = one.get((j, g), 0.0) + p * w
    tot = {(0, 0): 1.0}
    for _ in range(5):
        nxt = {}
        for (t, g), p in tot.items():
            for (j, h), q in one.items():
                key = (t + j, g + h)
                nxt[key] = nxt.get(key, 0.0) + p * q
        tot = nxt
    return tot

def pass_prob(m, e, review):
    s = 0.0
    for (t, g), p in joint(m, e).items():
        if review and g in (15, 16):
            s += p * (t >= 16)
        else:
            s += p * (g >= 16)
    return s

def metrics(e, review, n=400):
    pas = fp = fn = comp = 0.0
    for i in range(n):
        m = (i + 0.5) / n
        w = 168 * m ** 5 * (1 - m) ** 2 / n
        pp = pass_prob(m, e, review)
        pas += w * pp
        if m >= 0.80:
            comp += w
            fn += w * (1 - pp)
        else:
            fp += w * pp
    return pas, fp / pas, fn / comp

if __name__ == "__main__":
    print("     e  review   pass rate  false pass  competent fail")
    for e in (0.0, 0.1, 0.2, 0.3):
        for rv in (False, True):
            if e == 0.0 and rv:
                continue
            a, b, c = metrics(e, rv)
            print("   {:.1f} {:7s} {:11.4f} {:11.4f} {:15.4f}".format(
                e, "yes" if rv else "no", a, b, c))
    print("\nReading: a question marked 4 can only be marked down, so")
    print("grader noise mostly fails competent candidates and hardly")
    print("changes the share of weak passers. A human review of scripts")
    print("graded 15 or 16 recovers most of the damage. The exam's own")
    print("coarseness (e = 0) remains the larger error.")
#      e  review   pass rate  false pass  competent fail
#    0.0 no           0.3223      0.4930          0.1953
#    0.1 no           0.3145      0.4935          0.2156
#    0.1 yes          0.3211      0.4925          0.1977
#    0.2 no           0.3066      0.4944          0.2367
#    0.2 yes          0.3179      0.4914          0.2039
#    0.3 no           0.2986      0.4957          0.2585
#    0.3 yes          0.3134      0.4900          0.2131
#
# Reading: a question marked 4 can only be marked down, so
# grader noise mostly fails competent candidates and hardly
# changes the share of weak passers. A human review of scripts
# graded 15 or 16 recovers most of the damage. The exam's own
# coarseness (e = 0) remains the larger error.

一个出人意料的结果是:对称的阅卷噪声并不主要是放进了弱考生,而是拒掉了强考生。原因在于分数的边界:一道满分 4 分的题只能被判低,不能被判高。e=0.3e = 0.3 时,胜任者当天不及格的比例从 0.1953 升到 0.2585,而误放比例几乎不变(0.4957)。边界复核把胜任者的误拒降回 0.2131。

所以,AI 阅卷噪声在这里不是主要问题;5 道题本身的粗糙(e=0e = 0 时误放已是 0.4930)才是。边界复核是一个代价很小、效果明显的补丁。

应试教学

Campbell 曾指出,一个量化指标越是被用于社会决策,它就越容易受到腐蚀,越会扭曲它本来要监测的过程 (Campbell 1979)。考试也是这样:一旦牌照收入取决于考试,就会出现针对题库的辅导,分数与真实技能的相关随之下降。

程序用一个高斯模型描述这一过程:考试表现与真实技能的相关为 ρ\rho,及格线始终让 32.23% 的人通过(第三章的及格率),品牌价格沿用第五章的公式。

推导

# Chapter 6. [p13] Teaching to the test: when the score stops tracking skill.
# Skill z and exam aptitude x are standard normal with correlation rho:
# x = rho z + sqrt(1 - rho^2) u. Mastery is m = 0.667 + 0.15 z (about
# the Beta(6, 3) of p05). The pass mark is set so that 32.23% pass, the
# p05 pass rate. Coaching aimed at the item bank lowers rho. For each
# rho the program reports (200,000 draws, fixed seed) the mean mastery
# of passers, the share of passers below 0.80, and the brand price of
# p09, 80 exp(4 (Q - 0.75)). A remedy: add k unseen transfer questions
# whose own correlation with skill is 0.8, averaged with the old score.
import math
import random

def sim(rho, rng, n=200000, transfer=0, rt=0.8):
    xs, ms = [], []
    for _ in range(n):
        z = rng.gauss(0, 1)
        x = rho * z + math.sqrt(1 - rho * rho) * rng.gauss(0, 1)
        if transfer:
            t = sum(rt * z + math.sqrt(1 - rt * rt) * rng.gauss(0, 1)
                    for _ in range(transfer)) / transfer
            x = (x + transfer * t) / (1 + transfer)
        xs.append(x)
        ms.append(0.667 + 0.15 * z)
    cut = sorted(xs)[int(n * (1 - 0.3223))]
    pm = [m for x, m in zip(xs, ms) if x >= cut]
    q = sum(pm) / len(pm)
    low = sum(m < 0.80 for m in pm) / len(pm)
    return q, low, 80 * math.exp(4 * (q - 0.75))

if __name__ == "__main__":
    rng = random.Random(74013)
    print("   rho  transfer   mean m  below 0.80  brand price")
    for rho in (0.9, 0.7, 0.5, 0.3):
        q, low, p = sim(rho, rng)
        print("   {:.1f} {:9d} {:8.4f} {:11.4f} {:12.2f}".format(
            rho, 0, q, low, p))
    for k in (1, 3):
        q, low, p = sim(0.3, rng, transfer=k)
        print("   {:.1f} {:9d} {:8.4f} {:11.4f} {:12.2f}".format(
            0.3, k, q, low, p))
    print("\nReading: as coaching decouples the score from skill, the")
    print("pass pool drifts toward the population mean and the brand")
    print("premium that justified the mark melts away. Unseen transfer")
    print("questions - one corner shown, three to be found - restore")
    print("most of it.")
#    rho  transfer   mean m  below 0.80  brand price
#    0.9         0   0.8177      0.4561       104.89
#    0.7         0   0.7839      0.5603        91.60
#    0.5         0   0.7509      0.6422        80.27
#    0.3         0   0.7176      0.7158        70.28
#    0.3         1   0.7841      0.5591        91.68
#    0.3         3   0.8150      0.4650       103.75
#
# Reading: as coaching decouples the score from skill, the
# pass pool drifts toward the population mean and the brand
# premium that justified the mark melts away. Unseen transfer
# questions - one corner shown, three to be found - restore
# most of it.

ρ\rho 从 0.9 降到 0.3 时,及格者的平均掌握度从 0.8177 降到 0.7176,低于 0.80 的及格者从 0.4561 升到 0.7158,品牌价格从 104.89 降到 70.28。及格线还在,及格率也没变,但它过滤掉的已经不是低技能的人。

补救的办法是加入考生没有见过的迁移题:给出一个例子,要求考生自己推出其余情形。即使原来的题已经被辅导到 ρ=0.3\rho = 0.3,加一道相关为 0.8 的迁移题,平均掌握度回到 0.7841;加三道,回到 0.8150,接近 ρ=0.9\rho = 0.9 的水平。这正是第七章「举一隅不以三隅反」的意思。

假说 · 尚未验证

作者的诠释(尚未验证): 在作者的考核中,每科 5 道题里至少应有 3 道是题库外的新案例(只给方法,不给类似的例题),这能让持牌群体的质量在辅导市场出现后仍保持稳定。什么会证明这一判断错:若真实考试数据显示,题库题与新案例题的分数和执业后的客户评价相关程度相同,那么迁移题并没有更好地测出技能,这一要求可以取消。

小结

把客户一侧当作有限的池子之后,双边均衡的最优及格线是 18,16 分只少不到百分之一。AI 阅卷噪声主要伤害强考生,边界复核可以修补。应试教学会让及格线失去过滤作用,题库外的迁移题可以把它找回来。

参考文献

Armstrong, Mark. 2006. “Competition in Two-Sided Markets.” RAND Journal of Economics 37 (3): 668–91.
Campbell, Donald T. 1979. “Assessing the Impact of Planned Social Change.” Evaluation and Program Planning 2 (1): 67–90.
Rochet, Jean-Charles, and Jean Tirole. 2003. “Platform Competition in Two-Sided Markets.” Journal of the European Economic Association 1 (4): 990–1029.

Fan, Y. (2026). Diagnostic-to-Pedagogy Conversion Funnels, Mastery-Gated Curriculum Architecture and Certification Network Economics: Self-Cannibalization Bounds, Spaced-Retrieval Curves and Two-Sided Practitioner Licensing [诊学一体转化漏斗、掌握度门控课程与持牌人网络经济学] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090197 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).