What does Reasoning Trajectories and Outcome Luck conclude, in numbers, about “The Reasoning Graph: Nodes, Leaps, Backtracks and Revisions”?

Answer: At twelve edges the interval runs from 0.063 to 0.483, with a half-width of 0.210, which says almost nothing. At 48 edges it narrows to 0.111 to 0.333; at 96 edges it is 0.132 to 0.291; at 192 edges the half-width is 0.056; at 384 edges the interval runs from 0.163 to 0.243 and the half-width is 0.040.

Scope of the numbers: After-the-fact explanations are often folk theories: in a synthetic experiment, retrospective self-reports correlate only -0.013 with the weights used but 0.826 with an observer's guess; concurrent thinking aloud keeps order and leaps. Parameters are illustrative and all calculations come from the book's programs.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.040, 0.056, 0.063, 0.111, 0.132, 0.163

leap rate 0.20, Wilson 95% interval
edges low high half-width
120.063 0.483 0.210
240.087 0.396 0.154
480.111 0.333 0.111
960.132 0.291 0.079
1920.150 0.262 0.056
3840.163 0.243 0.040
edges per person to tell 0.15 from 0.30121
episodes of 12 edges11
within-episode icc 0.05: design effect 1.55, episodes 16
within-episode icc 0.10: design effect 2.10, episodes 22
Show the full program
# Chapter 3. p32: how many coded edges before a leap rate means much?
# Wilson 95 per cent interval for a leap rate of 0.20 at several edge
# counts; then the edges per person needed to tell a leap rate of 0.15
# from 0.30 (two-sided alpha 0.05, power 0.80, normal approximation for
# two proportions). One episode like p31 yields about a dozen edges.
# Edges inside one episode are not independent, so the counts below
# are lower bounds (see the design-effect line). Illustrative values.
import math
from statistics import NormalDist

Z = NormalDist()


def wilson(p, n, z=1.959964):
    den = 1 + z * z / n
    mid = (p + z * z / (2 * n)) / den
    half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
    return mid - half, mid + half


def n_two(p1, p2, alpha=0.05, power=0.80):
    za = Z.inv_cdf(1 - alpha / 2)
    zb = Z.inv_cdf(power)
    pb = (p1 + p2) / 2
    num = za * math.sqrt(2 * pb * (1 - pb)) + zb * math.sqrt(
        p1 * (1 - p1) + p2 * (1 - p2))
    return math.ceil((num / (p1 - p2)) ** 2)


def main():
    print("leap rate 0.20, Wilson 95% interval")
    print("edges   low    high   half-width")
    for n in (12, 24, 48, 96, 192, 384):
        lo, hi = wilson(0.20, n)
        print("{:>5}{:>7.3f}{:>8.3f}{:>9.3f}".format(n, lo, hi,
                                                      (hi - lo) / 2))
    n = n_two(0.15, 0.30)
    print("edges per person to tell 0.15 from 0.30: {}".format(n))
    print("episodes of 12 edges: {}".format(math.ceil(n / 12)))
    for icc in (0.05, 0.10):
        deff = 1 + (12 - 1) * icc
        print("within-episode icc {:.2f}: design effect {:.2f}, "
              "episodes {}".format(icc, deff, math.ceil(n * deff / 12)))


main()

Single source and citation

Fan, Y. (2026). Reasoning-Trajectory Profiling, Process-Tracing Games and Separating Cognitive Structure from Stated Intentions and Outcome Luck (Version 1.0.0), Chapter 3: The Reasoning Graph: Nodes, Leaps, Backtracks and Revisions. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-385 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026385,
  author    = {Fan, Yafu},
  title     = {Reasoning-Trajectory Profiling, Process-Tracing Games and Separating Cognitive Structure from Stated Intentions and Outcome Luck},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

Other chapter answers in this book