What does Reasoning Trajectories and Outcome Luck conclude, in numbers, about “The Reasoning Graph: Nodes, Leaps, Backtracks and Revisions”?
Answer: At twelve edges the interval runs from 0.063 to 0.483, with a half-width of 0.210, which says almost nothing. At 48 edges it narrows to 0.111 to 0.333; at 96 edges it is 0.132 to 0.291; at 192 edges the half-width is 0.056; at 384 edges the interval runs from 0.163 to 0.243 and the half-width is 0.040.
Scope of the numbers: After-the-fact explanations are often folk theories: in a synthetic experiment, retrospective self-reports correlate only -0.013 with the weights used but 0.826 with an observer's guess; concurrent thinking aloud keeps order and leaps. Parameters are illustrative and all calculations come from the book's programs.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.040, 0.056, 0.063, 0.111, 0.132, 0.163
| leap rate 0.20, Wilson 95% interval | |
| edges low high half-width | |
| 12 | 0.063 0.483 0.210 |
| 24 | 0.087 0.396 0.154 |
| 48 | 0.111 0.333 0.111 |
| 96 | 0.132 0.291 0.079 |
| 192 | 0.150 0.262 0.056 |
| 384 | 0.163 0.243 0.040 |
| edges per person to tell 0.15 from 0.30 | 121 |
| episodes of 12 edges | 11 |
| within-episode icc 0.05: design effect 1.55, episodes 16 | |
| within-episode icc 0.10: design effect 2.10, episodes 22 |
Show the full program
# Chapter 3. p32: how many coded edges before a leap rate means much?
# Wilson 95 per cent interval for a leap rate of 0.20 at several edge
# counts; then the edges per person needed to tell a leap rate of 0.15
# from 0.30 (two-sided alpha 0.05, power 0.80, normal approximation for
# two proportions). One episode like p31 yields about a dozen edges.
# Edges inside one episode are not independent, so the counts below
# are lower bounds (see the design-effect line). Illustrative values.
import math
from statistics import NormalDist
Z = NormalDist()
def wilson(p, n, z=1.959964):
den = 1 + z * z / n
mid = (p + z * z / (2 * n)) / den
half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
return mid - half, mid + half
def n_two(p1, p2, alpha=0.05, power=0.80):
za = Z.inv_cdf(1 - alpha / 2)
zb = Z.inv_cdf(power)
pb = (p1 + p2) / 2
num = za * math.sqrt(2 * pb * (1 - pb)) + zb * math.sqrt(
p1 * (1 - p1) + p2 * (1 - p2))
return math.ceil((num / (p1 - p2)) ** 2)
def main():
print("leap rate 0.20, Wilson 95% interval")
print("edges low high half-width")
for n in (12, 24, 48, 96, 192, 384):
lo, hi = wilson(0.20, n)
print("{:>5}{:>7.3f}{:>8.3f}{:>9.3f}".format(n, lo, hi,
(hi - lo) / 2))
n = n_two(0.15, 0.30)
print("edges per person to tell 0.15 from 0.30: {}".format(n))
print("episodes of 12 edges: {}".format(math.ceil(n / 12)))
for icc in (0.05, 0.10):
deff = 1 + (12 - 1) * icc
print("within-episode icc {:.2f}: design effect {:.2f}, "
"episodes {}".format(icc, deff, math.ceil(n * deff / 12)))
main()Single source and citation
Fan, Y. (2026). Reasoning-Trajectory Profiling, Process-Tracing Games and Separating Cognitive Structure from Stated Intentions and Outcome Luck (Version 1.0.0), Chapter 3: The Reasoning Graph: Nodes, Leaps, Backtracks and Revisions. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-385 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026385,
author = {Fan, Yafu},
title = {Reasoning-Trajectory Profiling, Process-Tracing Games and Separating Cognitive Structure from Stated Intentions and Outcome Luck},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Limits of Self-Report: After-the-Fact Reasons, Thinking Aloud and Reactivity
- Chapter 2 · Four Process-Tracing Tests Written as Likelihood Ratios
- Chapter 4 · Fingerprints under Pressure: Intraclass Correlation and Rater Agreement
- Chapter 5 · The Cost of Forgery: Probing Depth, Memory Load and Bounds on Detection
- Chapter 6 · Stripping Out Outcome Luck: Track Records, Sequential Tests and Multi-Method Composites
- Chapter 7 · Classical Sources: the Analects on Seeing, Watching and Examining; Bayes and Aristotle on Choice
- Chapter 8 · Field Tools and Checker Specifications: the Reasoning-Trajectory Selection Ledger, the Scorer, Pre-Registration and Refutation