What does Four-Pillar Computation conclude, in numbers, about “A Four-Step Check, an Agreement Study and a Pre-Registered Matching Test”?

Answer: Chapter 8 gives a four-step check, the sample size of an agreement study (255 charts fix kappa to within 0.100) and a pre-registered matching test (359 analysed participants). Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's hypotheses.

Scope of the numbers: Marked boxes separate published evidence, runnable Python calculations (16 programs), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.100, 255

   planning values: p_o 0.768, kappa 0.553, p_e 0.481
   half-width 0.100: 255 charts
   half-width 0.050: 1,017 charts
   half-width 0.025: 4,066 charts
Reading: a few hundred charts pin kappa down to within 0.1,
about a thousand to within 0.05. Agreement between engines can
be measured cheaply and should be published before any claim
about what a strong or weak chart means.
Show the full program
# Chapter 8. How many charts a concordance study needs.
# A concordance study gives the same charts, blind, to several engines
# or practitioners and measures agreement with Cohen's kappa. It tests
# reproducibility, not validity. With observed agreement p_o and chance
# agreement p_e, the large-sample standard error of kappa is about
#   se = sqrt(p_o (1 - p_o) / n) / (1 - p_e),
# so the n needed for a 95 % interval of half-width h follows. The
# planning values come from conventions A and B of p10.
import math

PO, KAPPA = 0.768, 0.553                  # A vs B in p10
PE = (PO - KAPPA) / (1 - KAPPA)

def n_for(h, po=PO, pe=PE, z=1.959964):
    return math.ceil(z * z * po * (1 - po) / ((1 - pe) ** 2 * h * h))

if __name__ == "__main__":
    print("   planning values: p_o {:.3f}, kappa {:.3f}, p_e {:.3f}".format(
        PO, KAPPA, PE))
    for h in (0.10, 0.05, 0.025):
        print("   half-width {:.3f}: {:,d} charts".format(h, n_for(h)))
    print("\nReading: a few hundred charts pin kappa down to within 0.1,")
    print("about a thousand to within 0.05. Agreement between engines can")
    print("be measured cheaply and should be published before any claim")
    print("about what a strong or weak chart means.")

Single source and citation

Fan, Y. (2026). Four-Pillar Calendrical Computation and Formal Rule Engines (Version 1.0.0), Chapter 8: A Four-Step Check, an Agreement Study and a Pre-Registered Matching Test. 5D Library. https://doi.org/10.5281/zenodo.24090180

DOI: 10.5281/zenodo.24090180 · Read the monograph · 5DL-B-066 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026066,
  author    = {Fan, Yafu},
  title     = {Four-Pillar Calendrical Computation and Formal Rule Engines},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.24090180},
  url       = {https://doi.org/10.5281/zenodo.24090180}
}

Other chapter answers in this book