5D Library › 对话式成交自动机、异议消解信息增益与高客单销售动力学 · English

第八章 一万段对话的账本、概率校准与预注册实验

一万段对话的账本

把第六章的模型逐个买家模拟 10,000 段对话:开始时只有 10% 的买家已经准备好(多数主动询问的对话都是冷的)。比较四种做法:最优的纯 AI 规则;最优的 AI 加人工交接规则;「一见到购买信号就请求付款」;「所有人一来就交给人」。退款按收入的 8% 扣除,每段 AI 对话成本 RM 3,每次人工交接 RM 250。

推导

# Chapter 8. p14: A ledger for 10,000 AI sales conversations (RM 9,800
# offer), simulated buyer by buyer with the model of p11; 10% of buyers
# are ready at the start (most inbound chats are cold). Playbooks: the
# optimal AI-only rule, the optimal rule with human hand-over (RM 250 a
# hand-over), "ask at the first buying signal", and "hand everyone to a
# human at once". Refunds take 8%
# of sales revenue; each AI conversation costs RM 3. Seed 84.
import importlib.util
import random

def load(name):
    spec = importlib.util.spec_from_file_location(name, name + ".py")
    m = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(m)
    return m

p11 = load("p11_stopping")
V = p11.V

def play(rng, rule, n=10000, p0=0.1):
    sold_ai = sold_h = hand = 0
    for _ in range(n):
        ready, p, sig = rng.random() < p0, p0, False
        for turn in range(60):
            a = rule(p, sig, turn)
            if a == "ask for payment":
                if ready and rng.random() < p11.ASK:
                    sold_ai += 1
                    break
                if not ready and rng.random() < p11.LEAVE:
                    break
                r, s = p * (1 - p11.ASK), (1 - p) * (1 - p11.LEAVE)
                p = r / (r + s)
                continue
            if a == "hand to a human":
                hand += 1
                sold_h += rng.random() < p11.HUMAN[0 if ready else 1]
                break
            if rng.random() < p11.DRIFT:
                break
            if not ready and rng.random() < p11.NURT:
                ready = True
            q = p + (1 - p) * p11.NURT
            y, n_ = p11.SIG
            sig = rng.random() < (y if ready else n_)
            p = q * y / (q * y + (1 - q) * n_) if sig else \
                q * (1 - y) / (q * (1 - y) + (1 - q) * (1 - n_))
    rev = (sold_ai + sold_h) * V
    net = rev * 0.92 - hand * 250 - n * 3
    return sold_ai, sold_h, hand, net

def policy(cost):
    _, act = p11.solve(cost)
    return lambda p, sig, t: act[min(p11.N, round(p * p11.N))]

if __name__ == "__main__":
    rules = {"AI only (optimal)": policy(1e9),
             "AI + human (optimal)": policy(250.0),
             "ask at first signal": lambda p, s, t: "ask for payment"
             if s else "keep talking",
             "everyone to a human": lambda p, s, t: "hand to a human"}
    print("   playbook               AI sales  human sales  hand-overs"
          "  net RM")
    for name, rule in rules.items():
        a, h, k, net = play(random.Random(84), rule)
        print("   {:<22} {:8,d}  {:11,d}  {:10,d}  {:>12,.0f}".format(
            name, a, h, k, net))
#    playbook               AI sales  human sales  hand-overs  net RM
#    AI only (optimal)         2,358            0           0    21,229,728
#    AI + human (optimal)          0        2,618       6,302    21,998,388
#    ask at first signal       1,930            0           0    17,370,880
#    everyone to a human           0        2,674      10,000    21,578,784

最优的 AI 加人工规则净收入 RM 21,998,388,交接 6,302 次、由人工成交 2,618 单;所有人一来就交给人的净收入是 RM 21,578,784,交接 10,000 次;最优的纯 AI 规则成交 2,358 单,净收入 RM 21,229,728;「一见信号就要钱」只成交 1,930 单,净收入 RM 17,370,880,是四者中最差的。在冷启动的对话中,买家的第一个购买信号就把信念推入交接区间,所以最优规则下 AI 的工作是筛选与培育,成交由人完成。

AI 说的「成交概率」是概率吗

第六章的阈值规则以 AI 给出的成交概率为输入。如果这个数字不是校准过的概率,阈值就会失效。下面模拟一个过度自信的评分:真实概率越极端,它说得越极端。

推导

# Chapter 8. p15: Is the AI closer's "probability of buying" a probability?
# 8,000 simulated conversations; the true chance t is Beta(2, 8); the
# model reports an overconfident score sigmoid(1.6 logit(t) + 0.3). The
# first 4,000 conversations fit a recalibration (mean outcome per score
# bin of width 0.1); the last 4,000 test it. Brier score = mean squared
# error. Also: how many test conversations cross the ask-for-payment
# threshold of p11 (0.5) on each score, and how many truly should. Seed 84.
import random
from math import exp, log

def sig(x):
    return 1 / (1 + exp(-x))

def logit(p):
    return log(p / (1 - p))

if __name__ == "__main__":
    rng = random.Random(84)
    rows = []
    for _ in range(8000):
        t = rng.betavariate(2, 8)
        s = sig(1.6 * logit(t) + 0.3)
        rows.append((t, s, 1 if rng.random() < t else 0))
    fit, test = rows[:4000], rows[4000:]
    bins = {}
    for t, s, y in fit:
        b = min(int(s * 10), 9)
        n, k = bins.get(b, (0, 0))
        bins[b] = (n + 1, k + y)
    cal = {b: k / n for b, (n, k) in bins.items()}

    def recal(s):
        b = min(int(s * 10), 9)
        return cal.get(b, s)
    brier = lambda f: sum((f(t, s) - y) ** 2 for t, s, y in test) / 4000
    print("   Brier score: raw {:.4f}, recalibrated {:.4f}, truth {:.4f}"
          .format(brier(lambda t, s: s), brier(lambda t, s: recal(s)),
                  brier(lambda t, s: t)))
    print("   score bin  conversations  mean score  share who bought")
    for b in range(0, 10, 2):
        sel = [(s, y) for t, s, y in test if b / 10 <= s < (b + 2) / 10]
        if sel:
            print("   {:.1f}-{:.1f}    {:13d}  {:10.3f}  {:16.3f}".format(
                b / 10, (b + 2) / 10, len(sel),
                sum(s for s, _ in sel) / len(sel),
                sum(y for _, y in sel) / len(sel)))
    over = [sum(1 for t, s, y in test if f(t, s) >= 0.5)
            for f in (lambda t, s: s, lambda t, s: recal(s),
                      lambda t, s: t)]
    print("   above the 0.5 threshold: raw {}, recalibrated {}, truly {}"
          .format(*over))
#    Brier score: raw 0.1452, recalibrated 0.1439, truth 0.1422
#    score bin  conversations  mean score  share who bought
#    0.0-0.2             2874       0.079             0.132
#    0.2-0.4              810       0.280             0.309
#    0.4-0.6              248       0.479             0.419
#    0.6-0.8               64       0.665             0.609
#    0.8-1.0                4       0.822             0.500
#    above the 0.5 threshold: raw 157, recalibrated 68, truly 83

在 4,000 段测试对话中,原始评分的布里尔分数是 0.1452,按评分分段重新校准后是 0.1439,真实概率本身是 0.1422 (Brier 1950)。差距看起来很小,但阈值决策差别很大:原始评分有 157 段越过 0.5 的请求付款阈值,校准后是 68 段,真正应当越过的是 83 段。原始评分会让 AI 在一倍左右的对话中过早要钱。

注

四步核查。 第一,诊断提问问到不能再改变决定为止,不按比特多少排序。第二,对四大异议先解码原因,「太贵了」先问澄清问题;禁用虚假截止日期与反复催促 (Parliament of Malaysia 1999)。第三,交给人的对话按交给人之后的期望成交额或增量排序,不按热度;新来源按吉廷斯指数多给一些人工。第四,AI 的成交概率每月用实际结果重新校准,并以退款与拒付率为护栏 (Kohavi et al. 2020)。

预注册实验

推导

# Chapter 8. p16: Sample sizes for a pre-registered test (two-sided alpha
# 0.05, power 0.80). (1) Close rate now: pitch at once 0.108 against one
# diagnostic question first 0.138 (p02). (2) Value per lead: RM 2,522
# against RM 2,748 (p03); a lead's value is lumpy (most buy nothing), sd
# about RM 7,000. (3) Net per AI conversation: AI only RM 2,123 against
# AI with human hand-over RM 2,200 (p14), sd about RM 4,000.
from math import ceil, sqrt

ZA, ZB = 1.96, 0.84

def two_props(p1, p2):
    pb = (p1 + p2) / 2
    num = (ZA * sqrt(2 * pb * (1 - pb)) +
           ZB * sqrt(p1 * (1 - p1) + p2 * (1 - p2))) ** 2
    return ceil(num / (p1 - p2) ** 2)

def two_means(m1, m2, sd):
    return ceil(2 * ((ZA + ZB) * sd / (m2 - m1)) ** 2)

if __name__ == "__main__":
    print("   (1) close rate 0.108 vs 0.138: {:,} leads per arm".format(
        two_props(0.108, 0.138)))
    print("   (2) value RM 2,522 vs 2,748: {:,} leads per arm".format(
        two_means(2522, 2748, 7000)))
    print("   (3) net RM 2,123 vs 2,200: {:,} conversations per arm".format(
        two_means(2123, 2200, 4000)))
#    (1) close rate 0.108 vs 0.138: 1,879 leads per arm
#    (2) value RM 2,522 vs 2,748: 15,043 leads per arm
#    (3) net RM 2,123 vs 2,200: 42,315 conversations per arm

检验「先问一个诊断问题」对当下成交率的影响(0.108 对 0.138),每组需要 1,879 位询价者;检验每位询价者的价值(RM 2,522 对 RM 2,748),因为价值很不均匀,每组需要 15,043 位;检验人工交接规则的净收入差异(每段 RM 2,123 对 RM 2,200),每组需要 42,315 段对话。

假说 · 尚未验证

预注册的预测(尚未验证): 先问一个情境问题再报价,比直接报价,当下成交率至少高 2 个百分点;按信念阈值交接,比所有人一来就交给人,每段对话的净收入不低。什么会证明它错: 前者的成交率差距显著小于 2 个百分点,或后者的净收入显著更低。

全书小结

高客单私域对话是一个部分可观测的决策问题。过早报价并非没有信息,它的代价是被拒绝之后的流失;诊断提问的价值在于改变决定,最优提问树平均不到两题,完全信息的价值上限比直接报价高 37%。四大异议各有不同的原因,按后验回应比一套话术多收回三成以上;高压话术计入退款、拒付与声誉之后净值为负。AI 与人工的分工由信任阈值停时规则决定,交给人的应当是人能多成交的那些,人工坐席按排队配置,AI 的概率要校准。

参考文献

Brier, Glenn W. 1950. “Verification of Forecasts Expressed in Terms of Probability.” Monthly Weather Review 78 (1): 1–3.
Kohavi, Ron, Diane Tang, and Ya Xu. 2020. Trustworthy Online Controlled Experiments: A Practical Guide to a/b Testing. Cambridge University Press.
Parliament of Malaysia. 1999. Consumer Protection Act 1999 (Act 599), Part II: Misleading and Deceptive Conduct, False Representation and Unfair Practice. Laws of Malaysia.

Fan, Y. (2026). Conversational Closing Automata, Objection-Handling Information Gain and High-Ticket Sales Kinetics: Diagnostic Question Trees, Trust-Threshold Stopping Rules and AI Closer Escalation Protocols [对话式成交自动机、异议消解信息增益与高客单销售动力学] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090221 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).