5D Library › 检索增强生成几何、倒数秩融合与多智能体误差级联 · English

第五章 验证器的独立性与重试的代价

第四章的两个隐含假设

第四章的漂亮结果——十步链从 43% 升到 99% 以上——依赖两个没有明说的假设。第一,重试是一次全新的抽样:上一次失败不影响下一次成功的机会。第二,验证器的错误与生成器的错误无关。本章放松这两个假设,并计算可靠性的另一项代价:尾部延迟。

难的步骤一直难

推导

# Chapter 5. p10: When the verifier shares the generator's blind spots.
# Ten-step chain, up to 3 attempts per step. Each step is either easy or
# hard, and a retry faces the same step, so a hard step stays hard:
#   easy step  generator right with 0.95
#   hard step  generator right with 0.40
# The share of hard steps h is chosen so a single attempt is right with
# 0.92 overall: h = 0.03 / 0.55. Two verifiers:
#   code check  exact on every step (s = 1, f = 0)
#   LLM judge   on easy steps catches 0.95 of errors, false alarms 0.05;
#               on hard steps it shares the generator's blind spot and
#               catches only c_hard of errors
# 100,000 simulated chains for each setting.
import random

H = 0.03 / 0.55

def run(rng, judge, c_hard, chains=100000):
    ok_all = esc = 0
    for _ in range(chains):
        ok = True
        stopped = False
        for _ in range(10):
            hard = rng.random() < H
            p = 0.40 if hard else 0.95
            done = False
            for _ in range(3):
                right = rng.random() < p
                if judge == "code":
                    passed = right
                else:
                    s = c_hard if hard else 0.95
                    passed = (rng.random() >= 0.05) if right else \
                        (rng.random() >= s)
                if passed:
                    ok = ok and right
                    done = True
                    break
            if not done:
                stopped = True
                break
        esc += stopped
        ok_all += ok and not stopped
    return ok_all / chains, esc / chains

if __name__ == "__main__":
    rng = random.Random(61010)
    print("share of hard steps h = {:.4f}; one attempt right with {:.4f}"
          .format(H, H * 0.40 + (1 - H) * 0.95))
    print("\n   verifier     catches on hard steps   success   escalated")
    s, e = run(rng, "code", 1.0)
    print("   code check            1.00              {:.4f}    {:.4f}"
          .format(s, e))
    for c in (0.95, 0.6, 0.3, 0.1):
        s, e = run(rng, "judge", c)
        print("   LLM judge             {:.2f}              {:.4f}    {:.4f}"
              .format(c, s, e))
    print("\nReading: retries assume each attempt is a fresh draw. Here a")
    print("hard step stays hard, so even an exact check reaches only")
    print("0.886 (p09's independent model gave 0.9949): what it cannot")
    print("fix it escalates instead of letting through. A judge that is as")
    print("good on hard steps as on easy ones comes close to it. One that")
    print("shares the generator's blind spot lets most hard-step errors")
    print("pass - fewer escalations, more silent failures. A verifier is")
    print("worth most where it is independent of the generator's")
    print("weakness.")
# share of hard steps h = 0.0545; one attempt right with 0.9200
#
#    verifier     catches on hard steps   success   escalated
#    code check            1.00              0.8860    0.1140
#    LLM judge             0.95              0.8342    0.1153
#    LLM judge             0.60              0.7656    0.0379
#    LLM judge             0.30              0.7200    0.0129
#    LLM judge             0.10              0.6950    0.0083
#
# Reading: retries assume each attempt is a fresh draw. Here a
# hard step stays hard, so even an exact check reaches only
# 0.886 (p09's independent model gave 0.9949): what it cannot
# fix it escalates instead of letting through. A judge that is as
# good on hard steps as on easy ones comes close to it. One that
# shares the generator's blind spot lets most hard-step errors
# pass - fewer escalations, more silent failures. A verifier is
# worth most where it is independent of the generator's
# weakness.

这里每一步要么容易(单次正确 0.95),要么困难(单次正确 0.40),困难步骤的比例 0.0545 使得单次尝试的总正确率仍是 0.9200,与第四章相同。区别在于:重试面对的是同一个步骤,困难的步骤一直困难。

即使是确定性的代码检查,成功率也只有 0.8860,而第四章独立模型给出的是 0.9949:困难步骤三次都做不对的概率不小,代码检查把这些情形升级(0.1140),而不是放过。语言模型评判在困难步骤上若与容易步骤一样敏锐(0.95),成功率 0.8342,接近代码检查。但若它与生成器共享盲区——在困难步骤上只能抓住一成的错误——成功率降到 0.6950:升级少了(0.0083),静默失败多了。

这是第二处对提纲的修正:验证器的价值不在于它平均有多准,而在于它在生成器最弱的地方是否独立。用同一个模型既生成又评判,最容易出现的正是这种共享盲区 (Zheng et al. 2023)。重试也不是免费的独立抽样:对真正困难的步骤,更换方法(换模型、换提示、交给人)比重复同一方法更有效。

重试预算与尾部延迟

推导

# Chapter 5. p11: Retry budget, cost and tail latency.
# Ten-step chain with an exact checker (as in p09), single attempts right
# with p. Up to n attempts per step; each attempt costs one unit (tokens
# or seconds). Exact distribution of total attempts by convolution of
# the per-step truncated geometric law; the run escalates if any step
# uses up its n attempts. Reported: success, mean attempts, and the 95th
# and 99th percentiles of attempts among successful runs.
def step_law(p, n):
    # P(step passes on attempt j), j = 1..n
    return [p * (1 - p) ** (j - 1) for j in range(1, n + 1)]

def chain_law(p, n, steps=10):
    dist = {0: 1.0}
    law = step_law(p, n)
    for _ in range(steps):
        nxt = {}
        for tot, pr in dist.items():
            for j, q in enumerate(law, 1):
                nxt[tot + j] = nxt.get(tot + j, 0.0) + pr * q
        dist = nxt
    return dist

def pct(dist, q):
    tot = sum(dist.values())
    acc = 0.0
    for k in sorted(dist):
        acc += dist[k] / tot
        if acc >= q:
            return k
    return max(dist)

if __name__ == "__main__":
    print("     p    n   success   mean attempts   p95   p99")
    for p in (0.92, 0.80, 0.60):
        for n in (1, 2, 3, 5):
            d = chain_law(p, n)
            succ = sum(d.values())
            mean = sum(k * v for k, v in d.items()) / succ
            print("   {:.2f}  {:2d}   {:.4f}      {:6.2f}         {:3d}"
                  "   {:3d}".format(p, n, succ, mean, pct(d, 0.95),
                                    pct(d, 0.99)))
    print("\nReading: with a strong generator the second and third")
    print("attempts buy almost all the reliability for little extra cost")
    print("or tail. With a weak one, each extra allowed attempt still")
    print("raises success substantially, but the 99th-percentile run")
    print("grows by several attempts - the price of reliability is paid")
    print("in the tail, and a latency budget should be set on it.")
#      p    n   success   mean attempts   p95   p99
#    0.92   1   0.4344       10.00          10    10
#    0.92   2   0.9378       10.74          12    13
#    0.92   3   0.9949       10.85          13    14
#    0.92   5   1.0000       10.87          13    14
#    0.80   1   0.1074       10.00          10    10
#    0.80   2   0.6648       11.67          14    15
#    0.80   3   0.9228       12.26          15    16
#    0.80   5   0.9968       12.48          16    17
#    0.60   1   0.0060       10.00          10    10
#    0.60   2   0.1749       12.86          15    16
#    0.60   3   0.5161       14.62          18    20
#    0.60   5   0.9022       16.15          21    24
#
# Reading: with a strong generator the second and third
# attempts buy almost all the reliability for little extra cost
# or tail. With a weak one, each extra allowed attempt still
# raises success substantially, but the 99th-percentile run
# grows by several attempts - the price of reliability is paid
# in the tail, and a latency budget should be set on it.

有确定性检查时,可靠性与成本的关系取决于生成器的强弱。单步 0.92 时,每步允许两次就把成功率从 0.4344 提到 0.9378,三次到 0.9949;平均尝试次数几乎不变(10.00 到 10.85),第 99 百分位从 10 次增加到 14 次。单步只有 0.60 时,每多允许一次仍大幅提高成功率(两次 0.1749,三次 0.5161,五次 0.9022),但平均尝试从 10.00 涨到 16.15,第 99 百分位涨到 24 次。

可靠性的代价主要付在尾部:多数运行很快,少数运行要反复重试。对需要响应时间保证的系统,重试上限应当按尾部延迟(例如第 99 百分位)来定,而不是按平均成本。

每步最多尝试 n 次时,十步链的成功率与第 99 百分位尝试次数,三种单步正确率。

注

这些数字的适用范围。 本章的「容易/困难」二分与固定的单步正确率都是简化。真实任务中,难度是连续的,且与输入有关;测量自己的系统时,应当按步骤与输入类型分别记录首次正确率、重试后正确率与验证器的检出率,而不是只记一个平均数。

假说 · 尚未验证

作者的假说(尚未验证): 在用同一个语言模型既生成又评判的智能体系统中,评判在「生成器首次失败的步骤」上的检出率,显著低于它在随机注入错误上的检出率。什么结果会证明它错:在预先登记的错误集上,两种检出率之差的 95% 置信区间包含零或为负。

小结

重试默认每次都是新的抽样,但困难的步骤一直困难:即使有确定的检查,十步链也只能达到 0.886,余下的由升级兜底。与生成器共享盲区的评判放过最难的错误,升级变少、静默失败变多。可靠性的代价付在尾部:生成器越弱,换取同样成功率所需的重试越多,第 99 百分位的延迟越长。

参考文献

Zheng, Lianmin, Wei-Lin Chiang, Ying Sheng, et al. 2023. “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.” Advances in Neural Information Processing Systems 36 (NeurIPS), Datasets and Benchmarks Track.

Fan, Y. (2026). Retrieval-Augmented Generation Geometry, Reciprocal Rank Fusion and Multi-Agent Error Cascades: High-Dimensional Concentration of Measure, Hybrid Lexical-Dense Retrieval Bounds and Verifier-Gated Fixed Points [检索增强生成几何、倒数秩融合与多智能体误差级联] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090170 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).