5D Library › 诊学一体转化漏斗、掌握度门控课程与持牌人网络经济学 · English

第八章 课程与牌照的审计,以及一项预注册实验

五个问题

本章把全书归结为五个问题,可以拿来检查任何一套「测算—课程—牌照」设计,包括作者自己的。它们不是经营建议。

一:课程价是否低于买家放弃的零售现值? 若是,课程在买家身上自蚕食,只能靠牌照漏斗补回(第一章)。

二:题库是否少于每允许一次考试 20 道题? 若是,补考会被记题攻破(第三章)。

三:是否每章都考? 若是,门槛与疲劳叠加,完课率大降(第四章)。

四:及格线是否低于 14/20? 若是,声誉会被侵蚀,双边网络中的通过者也会因分不到客户而不领牌(第五、六章)。

五:考试中是否没有题库外的迁移题? 若是,应试教学会让及格线失去过滤作用(第六、七章)。

审计程序

程序把五个问题写成五个自动标志,用于两套设计:一套是常见的「看完即发证」快速设计(课程价 180,题库 40 题,允许考 5 次,每章都考,及格线 12,无迁移题),另一套是本书推导出的门控阶梯(课程价 480,题库 400 题,允许考 4 次,按科目设门,及格线 16,每科 3 道迁移题)。

推导

# Chapter 8. [p15] Five automatic flags for a course-and-licence design.
# F1 course price below the retail value its buyers give up (p02);
# F2 item bank under 20 questions per allowed attempt (p06);
# F3 an exam after every chapter rather than every subject (p07);
# F4 pass mark below 14 of 20 (p10, p11: reputation erodes);
# F5 no unseen transfer questions in the exam (p13).
# Two designs are audited: a quick "watch and certify" design and the
# gated ladder developed in this book.

DESIGNS = {
    "quick": dict(price=180, buyers_pv=275.82, bank=40, attempts=5,
                  gate="chapter", mark=12, transfer=0),
    "ladder": dict(price=480, buyers_pv=499.25, bank=400, attempts=4,
                   gate="subject", mark=16, transfer=3),
}

def flags(d):
    return [
        ("F1 course below buyers' retail PV", d["price"] < d["buyers_pv"]),
        ("F2 bank < 20 per attempt", d["bank"] < 20 * d["attempts"]),
        ("F3 exam after every chapter", d["gate"] == "chapter"),
        ("F4 pass mark below 14/20", d["mark"] < 14),
        ("F5 no transfer questions", d["transfer"] == 0),
    ]

if __name__ == "__main__":
    for name, d in DESIGNS.items():
        fl = flags(d)
        n = sum(f for _, f in fl)
        print("   {} design: {} of 5 flags".format(name, n))
        for label, f in fl:
            print("     [{}] {}".format("X" if f else " ", label))
    print("\nReading: the quick design trips every flag. The ladder")
    print("still trips F1: priced alone the course is below what its")
    print("buyers would have spent on readings, so it is justified")
    print("only if the licence funnel of p01 is real - which is what")
    print("the experiment of p16 must show.")
#    quick design: 5 of 5 flags
#      [X] F1 course below buyers' retail PV
#      [X] F2 bank < 20 per attempt
#      [X] F3 exam after every chapter
#      [X] F4 pass mark below 14/20
#      [X] F5 no transfer questions
#    ladder design: 1 of 5 flags
#      [X] F1 course below buyers' retail PV
#      [ ] F2 bank < 20 per attempt
#      [ ] F3 exam after every chapter
#      [ ] F4 pass mark below 14/20
#      [ ] F5 no transfer questions
#
# Reading: the quick design trips every flag. The ladder
# still trips F1: priced alone the course is below what its
# buyers would have spent on readings, so it is justified
# only if the licence funnel of p01 is real - which is what
# the experiment of p16 must show.

快速设计五个标志全部触发。门控阶梯只触发第一个:单看课程本身,480 的价格低于其买家放弃的零售现值 499.25(第一章)。这不是疏忽,而是本书的核心判断:课程只有在牌照漏斗真实存在时才划算。这正是下面的实验必须检验的。

一项预注册实验

假说 · 尚未验证

研究方案。 报名学员按个人随机分入「开放」组(看完即完成)或「门控」组(科目考试、个人化间隔复习、迁移题与未通过后的矫正单元)。主要结局:第 90 天在一组从未见过的案例上的标准化得分,预期效应量 d=0.30d = 0.30。必须报告的伤害:12 个科目的完课率(模型预期 0.5307 对 0.1656)。长期登记:持牌后每次服务的出问题率(及格线 14 对 16,模型预期 0.00471 对 0.00333)。检验:双侧 0.05,把握度 0.90 (Cohen 1988)。由与作者无经济关系的独立分析者按预先登记的代码执行;结果不论正负都发表。

推导

# Chapter 8. [p16] Sample sizes for a pre-registered course experiment.
# Enrolees are randomised to OPEN (watch and finish) or GATE (subject
# exams, spaced review, transfer questions). Outcomes, two-sided 0.05,
# power 0.90, normal approximation:
# (a) primary: score on an unseen case set at day 90, standardised
#     effect d = 0.30;
# (b) completion of all 12 subjects, 0.5307 vs 0.1656 (p08) - a harm
#     the protocol must report, not hide;
# (c) bad sessions per session among licensees, 0.00471 (mark 14) vs
#     0.00333 (mark 16), p09 - counted in sessions.
import math
from statistics import NormalDist

def n_means(d, alpha=0.05, power=0.90):
    z = NormalDist().inv_cdf
    return math.ceil(2 * ((z(1 - alpha / 2) + z(power)) / d) ** 2)

def n_props(p1, p2, alpha=0.05, power=0.90):
    z = NormalDist().inv_cdf
    pb = (p1 + p2) / 2
    a = z(1 - alpha / 2) * math.sqrt(2 * pb * (1 - pb))
    b = z(power) * math.sqrt(p1 * (1 - p1) + p2 * (1 - p2))
    return math.ceil((a + b) ** 2 / (p1 - p2) ** 2)

if __name__ == "__main__":
    print("   (a) enrolees per arm, d = 0.30      : {:,d}".format(
        n_means(0.30)))
    print("   (b) enrolees per arm, completion    : {:,d}".format(
        n_props(0.5307, 0.1656)))
    ns = n_props(0.00471, 0.00333)
    print("   (c) sessions per arm, bad sessions  : {:,d}".format(ns))
    print("       = licensee-years at 100 sessions: {:,d}".format(
        math.ceil(ns / 100)))
    print("\nReading: the learning outcome needs a few hundred enrolees")
    print("per arm; the harm to completion shows with a few dozen. The")
    print("client-safety outcome needs tens of thousands of sessions")
    print("per arm - hundreds of licensee-years - and can only be a")
    print("long-run registry, not a trial.")
#    (a) enrolees per arm, d = 0.30      : 234
#    (b) enrolees per arm, completion    : 34
#    (c) sessions per arm, bad sessions  : 44,180
#        = licensee-years at 100 sessions: 442
#
# Reading: the learning outcome needs a few hundred enrolees
# per arm; the harm to completion shows with a few dozen. The
# client-safety outcome needs tens of thousands of sessions
# per arm - hundreds of licensee-years - and can only be a
# long-run registry, not a trial.

学习结局每组需要 234 位学员;完课率的差别很大,每组 34 人就能看出。客户安全结局则需要每组 44,180 次服务,按每位持牌人每年 100 次计约 442 个「持牌人年」:它不能是一次实验,只能是长期登记。方案因此把学习结局定为主要结局,把完课率作为必须报告的伤害,把客户安全作为长期登记,而不是让一项样本不足的实验报告「没有差别」。

小结

五个问题——课程是否自蚕食、题库是否够大、门槛是否设在科目、及格线是否不低于 14、是否有迁移题——可以写成自动审计。检验门控阶梯的实验应以第 90 天的迁移测验为主要结局,每组 234 人;完课率的下降必须如实报告;客户安全只能靠长期登记。

参考文献

Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum.

Fan, Y. (2026). Diagnostic-to-Pedagogy Conversion Funnels, Mastery-Gated Curriculum Architecture and Certification Network Economics: Self-Cannibalization Bounds, Spaced-Retrieval Curves and Two-Sided Practitioner Licensing [诊学一体转化漏斗、掌握度门控课程与持牌人网络经济学] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090197 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).