What does A Zero-Leak Socratic Examiner conclude, in numbers, about “Physical Isolation: Server-Side Grading, Information-Flow Graphs and the One-Bit Oracle”?
Answer: Program p21 draws four architectures as information-flow graphs. Only architecture C, with grading on the server and only bits for the model, pinches every path to the browser down to one bit, and a single hint writer that reads the answer is enough to turn it back into D. Program p22 shows that grading is itself an oracle that reveals at most one bit per verdict and offers almost no protection to small-space items such as four-option questions. Program p23 shows that caps and parallel items hold the oracle down, but that small-space items cannot guard against guessing and against false lockouts at the same time. With the content channel closed, the examiner faces one question: holding nothing but a string of rights and wrongs, on what grounds can it judge that a learner has mastered the skill? The next chapter introduces Bayesian knowledge tracing.
Scope of the numbers: Parameters are illustrative; all calculations are generated by the book's programs, which call no model interface, contain no keys or server addresses and include no concrete extraction prompts. This is not individual business advice.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python only
| answer space 1000 values = 9.9658 bits | |||||
| arch paths full bit bits/reply worst path | |||||
| A | 1 | 1 | 0 | 9.9658 | key>ctx>out>browser |
| B | 1 | 1 | 0 | 9.9658 | key>bundle>browser |
| C | 1 | 0 | 1 | <= 1 | key>grader>verdict>ctx>out>browser |
| D | 2 | 1 | 1 | 9.9658 | key>hinter>ctx>out>browser |
| only C keeps every path from key to browser behind a bit |
Show the full program
# Chapter 2. p21: where can the answer flow? Reachability on a graph.
# Each architecture is a list of directed channels. A channel either
# carries text that can contain the answer ("full") or only a verdict
# bit, correct or not ("bit"). We ask whether the answer key can reach
# anything the student sees (the browser), and through what kind of
# channel. Node names are generic; no real system is described.
# A answer pasted into the model's prompt with "do not reveal it"
# B answers shipped to the page so the browser grades itself
# C server-side grader; the model gets only the verdict bit
# D like C, but a hint writer that reads the answer feeds the model
import math
ARCH = (
("A", (("key", "ctx", "full"), ("ctx", "out", "full"),
("out", "browser", "full"), ("student", "ctx", "full"))),
("B", (("key", "bundle", "full"), ("bundle", "browser", "full"),
("student", "browser", "full"))),
("C", (("key", "grader", "full"), ("student", "grader", "full"),
("grader", "verdict", "bit"), ("verdict", "ctx", "bit"),
("student", "ctx", "full"), ("ctx", "out", "full"),
("out", "browser", "full"))),
("D", (("key", "grader", "full"), ("grader", "verdict", "bit"),
("verdict", "ctx", "bit"), ("key", "hinter", "full"),
("hinter", "ctx", "full"), ("ctx", "out", "full"),
("out", "browser", "full"))),
)
def worst_paths(edges, src, dst):
# depth-first search over simple paths, in the listed edge order
found = []
def walk(node, path, kind):
if node == dst:
found.append((kind, path))
return
for a, b, k in edges:
if a == node and b not in path:
nk = "full" if kind == "full" and k == "full" else "bit"
walk(b, path + [b], nk)
walk(src, [src], "full")
return found
SPACE = 1000
print("answer space %d values = %.4f bits" % (SPACE, math.log2(SPACE)))
print("arch paths full bit bits/reply worst path")
for name, edges in ARCH:
ps = worst_paths(edges, "key", "browser")
n_full = len([1 for k, _ in ps if k == "full"])
n_bit = len(ps) - n_full
if n_full:
bits = "%.4f" % math.log2(SPACE)
elif n_bit:
bits = "<= 1"
else:
bits = "0"
full = [p for k, p in ps if k == "full"]
worst = full[0] if full else (ps[0][1] if ps else [])
shown = ">".join(worst) if worst else "none"
print("%-4s %-5d %-4d %-3d %-10s %s"
% (name, len(ps), n_full, n_bit, bits, shown))
print("only C keeps every path from key to browser behind a bit")Single source and citation
Fan, Y. (2026). A Zero-Leak Socratic Examiner: Server-Side Isolation of Answers and Bayesian Mastery Probing Trees (Version 1.0.0), Chapter 2: Physical Isolation: Server-Side Grading, Information-Flow Graphs and the One-Bit Oracle. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-262 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026262,
author = {Fan, Yafu},
title = {A Zero-Leak Socratic Examiner: Server-Side Isolation of Answers and Bayesian Mastery Probing Trees},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Telling It Not to Tell: Answers in Context and the Cumulative Leak Probability
- Chapter 3 · Bayesian Knowledge Tracing: Four Parameters, the Mastery Line and Identifiability
- Chapter 4 · How Leaks Forge Mastery: Guess Rates, False Certificates and the Blind Spot of Fitting
- Chapter 5 · The Probing Tree: Information-Gain Item Selection, Greedy Trees and Stopping Rules
- Chapter 6 · Defence in Depth against Prompt Extraction: Correlated Layer Failure, Digest-Only Output Checks and Item Exposure
- Chapter 7 · Classical Sources: "No Opening without Striving" in the Analects and the Slave-Boy Geometry of Plato's Meno
- Chapter 8 · The Empirical Loop: Red-Teaming the Academy Examiner for Leaks, the Mastery Probing Tree and a Pre-Registered Test