What does Reasoning Trajectories and Outcome Luck conclude, in numbers, about “Field Tools and Checker Specifications: the Reasoning-Trajectory Selection Ledger, the Scorer, Pre-Registration and Refutation”?
Answer: Chapter 8 gives field tools and checker specifications: ledger fields, five gates, the distilled Reasoning-Trajectory Process-Tracing and Outcome-Luck Separation Scorer, checks grouped by the four kits, and a pre-registered plan whose binding validity test needs 347 people. All reasoning traces are collected with prior notice and consent; the book describes no covert lie detection, inducement, deception or extraction of trade secrets, uses no protected characteristic, and keeps outcomes in a separate column that never enters the reasoning score. Marked boxes separate published evidence, runnable Python calculations (21 programs), classical sources and the author's hypotheses.
Scope of the numbers: After-the-fact explanations are often folk theories: in a synthetic experiment, retrospective self-reports correlate only -0.013 with the weights used but 0.826 with an observer's guess; concurrent thinking aloud keeps order and leaps. Parameters are illustrative and all calculations come from the book's programs.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 347
| T1 validity r 0.25: n | 124 r 0.15: n = 347 |
| T2 increment Delta R^2 0.0347, f^2 0.0482, n | 167 |
| T3 depth 6 | 13 per group depth 4: 188 per group |
| T4 ICC 0.40 vs 0.60, k | 3: n = 53 people |
| binding size (T1 at r 0.15) | 347 people with 6-month follow-up |
Show the full program
# Chapter 8. p82: sample sizes for the pre-registered tests.
# T1 validity: reasoning score against the six-month delayed record,
# Fisher z, two-sided alpha 0.05, power 0.80, r = 0.25 and 0.15.
# T2 increment over structured conversation and work sample:
# Delta R^2 from p63 at rho 0.1 (0.528 vs 0.494), Cohen f^2, normal
# approximation n = (za + zb)^2 / f^2 + predictors + 1.
# T3 forgery cost: flag rate of consented role-play performers against
# genuine reasoners at depth 6 (0.784 vs 0.241) and depth 4 (0.218
# vs 0.111), two proportions, per group.
# T4 stability: ICC(3,1) over three scenes, H0 0.40 against 0.60,
# Fisher z for the intraclass correlation, one-sided alpha 0.05.
import math
from statistics import NormalDist
Z = NormalDist().inv_cdf
def n_corr(r, alpha=0.05, power=0.80):
return math.ceil(((Z(1 - alpha / 2) + Z(power)) / math.atanh(r)) ** 2
+ 3)
def n_two(p1, p2, alpha=0.05, power=0.80):
pb = (p1 + p2) / 2
num = Z(1 - alpha / 2) * math.sqrt(2 * pb * (1 - pb)) + Z(power) * \
math.sqrt(p1 * (1 - p1) + p2 * (1 - p2))
return math.ceil((num / (p1 - p2)) ** 2)
def zicc(rho, k):
return 0.5 * math.log((1 + (k - 1) * rho) / (1 - rho))
def n_icc(r0, r1, k, alpha=0.05, power=0.80):
d = zicc(r1, k) - zicc(r0, k)
return math.ceil(2 + k / (2 * (k - 1)) *
((Z(1 - alpha) + Z(power)) / d) ** 2)
def main():
print("T1 validity r 0.25: n = {} r 0.15: n = {}".format(
n_corr(0.25), n_corr(0.15)))
r2_full, r2_two = 0.528 ** 2, 0.494 ** 2
f2 = (r2_full - r2_two) / (1 - r2_full)
n = math.ceil((Z(0.975) + Z(0.80)) ** 2 / f2 + 3 + 1)
print("T2 increment Delta R^2 {:.4f}, f^2 {:.4f}, n = {}".format(
r2_full - r2_two, f2, n))
print("T3 depth 6: {} per group depth 4: {} per group".format(
n_two(0.784, 0.241), n_two(0.218, 0.111)))
print("T4 ICC 0.40 vs 0.60, k = 3: n = {} people".format(
n_icc(0.40, 0.60, 3)))
total = n_corr(0.15)
print("binding size (T1 at r 0.15): {} people with 6-month "
"follow-up".format(total))
main()Single source and citation
Fan, Y. (2026). Reasoning-Trajectory Profiling, Process-Tracing Games and Separating Cognitive Structure from Stated Intentions and Outcome Luck (Version 1.0.0), Chapter 8: Field Tools and Checker Specifications: the Reasoning-Trajectory Selection Ledger, the Scorer, Pre-Registration and Refutation. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-385 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026385,
author = {Fan, Yafu},
title = {Reasoning-Trajectory Profiling, Process-Tracing Games and Separating Cognitive Structure from Stated Intentions and Outcome Luck},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · The Limits of Self-Report: After-the-Fact Reasons, Thinking Aloud and Reactivity
- Chapter 2 · Four Process-Tracing Tests Written as Likelihood Ratios
- Chapter 3 · The Reasoning Graph: Nodes, Leaps, Backtracks and Revisions
- Chapter 4 · Fingerprints under Pressure: Intraclass Correlation and Rater Agreement
- Chapter 5 · The Cost of Forgery: Probing Depth, Memory Load and Bounds on Detection
- Chapter 6 · Stripping Out Outcome Luck: Track Records, Sequential Tests and Multi-Method Composites
- Chapter 7 · Classical Sources: the Analects on Seeing, Watching and Examining; Bayes and Aristotle on Choice