What does The Algorithmic Mirror conclude, in numbers, about “Practice: A Prompt Audit and a Pre-registered Trial”?

Answer: The protocol plans for the middle case, 53 questions, and samples 60 to allow for questions dropped because a model refuses or a rater cannot assess them. The standard deviation of 1.0 rating point for within-question differences is an assumption to be checked in a 6-question pilot.

Scope of the numbers: Marked boxes separate published evidence, runnable Python calculations (14 programs, all checked), classical sources and the author's hypotheses.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 1.0, 53

SD of within-topic difference 0.835 topics
SD of within-topic difference 1.053 topics
SD of within-topic difference 1.275 topics
Show the full program
import math
from statistics import NormalDist

z = NormalDist().inv_cdf

def topics_needed(sd_diff, delta, alpha=0.025, power=0.90):
    """Paired design: each research topic gets both prompt types. alpha = 0.025 per
    co-primary outcome (novelty, rigour), a Bonferroni split of 0.05."""
    za, zb = z(1 - alpha / 2), z(power)
    return math.ceil(((za + zb) * sd_diff / delta) ** 2 + za ** 2 / 2)

# Outcomes: mean of three blinded experts' ratings on a 1–7 scale.
SESOI = 0.5
for sd in (0.8, 1.0, 1.2):
    print(f"SD of within-topic difference {sd}: {topics_needed(sd, SESOI)} topics")

Single source and citation

Fan, Y. (2026). The Algorithmic Mirror (Version 1.0.0), Chapter 8: Practice: A Prompt Audit and a Pre-registered Trial. 5D Library. https://doi.org/10.5281/zenodo.23723557

DOI: 10.5281/zenodo.23723557 · Read the monograph · 5DL-B-011 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026011,
  author    = {Fan, Yafu},
  title     = {The Algorithmic Mirror},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  doi       = {10.5281/zenodo.23723557},
  url       = {https://doi.org/10.5281/zenodo.23723557}
}