What does The Algorithmic Mirror conclude, in numbers, about “Practice: A Prompt Audit and a Pre-registered Trial”?
Answer: The protocol plans for the middle case, 53 questions, and samples 60 to allow for questions dropped because a model refuses or a rater cannot assess them. The standard deviation of 1.0 rating point for within-question differences is an assumption to be checked in a 6-question pilot.
Scope of the numbers: Marked boxes separate published evidence, runnable Python calculations (14 programs, all checked), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 1.0, 53
| SD of within-topic difference 0.8 | 35 topics |
| SD of within-topic difference 1.0 | 53 topics |
| SD of within-topic difference 1.2 | 75 topics |
Show the full program
import math
from statistics import NormalDist
z = NormalDist().inv_cdf
def topics_needed(sd_diff, delta, alpha=0.025, power=0.90):
"""Paired design: each research topic gets both prompt types. alpha = 0.025 per
co-primary outcome (novelty, rigour), a Bonferroni split of 0.05."""
za, zb = z(1 - alpha / 2), z(power)
return math.ceil(((za + zb) * sd_diff / delta) ** 2 + za ** 2 / 2)
# Outcomes: mean of three blinded experts' ratings on a 1–7 scale.
SESOI = 0.5
for sd in (0.8, 1.0, 1.2):
print(f"SD of within-topic difference {sd}: {topics_needed(sd, SESOI)} topics")Single source and citation
Fan, Y. (2026). The Algorithmic Mirror (Version 1.0.0), Chapter 8: Practice: A Prompt Audit and a Pre-registered Trial. 5D Library. https://doi.org/10.5281/zenodo.23723557
DOI: 10.5281/zenodo.23723557 · Read the monograph · 5DL-B-011 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026011,
author = {Fan, Yafu},
title = {The Algorithmic Mirror},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
doi = {10.5281/zenodo.23723557},
url = {https://doi.org/10.5281/zenodo.23723557}
}