What does A Zero-Leak Socratic Examiner conclude, in numbers, about “Defence in Depth against Prompt Extraction: Correlated Layer Failure, Digest-Only Output Checks and Item Exposure”?
Answer: Program p61 shows that the layers of defence in depth can be multiplied only when they fail independently, whereas the failure mechanisms revealed by current research make them fail together; only isolation moves the limit from 1 to 0. Program p62 demonstrates a digest-only output check: in the isolated architecture it guards against regressions and against the model computing the answer by itself, and it stops written variants but not rewording. Program p63 estimates the one channel isolation cannot close, learners sharing the items they know they answered correctly, and shows the role of variants and of refreshing the bank. With the account settled, an older question comes next: how did Socrates and Confucius, long before any server, handle asking without telling? The next chapter returns to the classical sources.
Scope of the numbers: Parameters are illustrative; all calculations are generated by the book's programs, which call no model interface, contain no keys or server addresses and include no concrete extraction prompts. This is not individual business advice.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it.
| layers | rho | per attempt | k=1000 | k=10000 | vs naive |
| base | 0.00 | 0.020000 | 1.0000 | 1.0000 | 33.33 |
| instruction | 0.00 | 0.006000 | 0.9976 | 1.0000 | 10.00 |
| filter | 0.00 | 0.000600 | 0.4513 | 0.9975 | 1.00 |
| correlated | 0.05 | 0.001370 | 0.7461 | 1.0000 | 2.28 |
| correlated | 0.10 | 0.002140 | 0.8826 | 1.0000 | 3.57 |
| isolated | 0.00 | 0.000000 | 0.0000 | 0.0000 | 0 |
| naive product of layers: 0.000600 per attempt | |||||
| isolation is the only row whose limit is 0, not 1 |
Show the full program
# Chapter 6. p61: layered defences multiply only if they fail apart.
# Per-attempt leak probabilities (all illustrative):
# base answer in context, no instruction p0 = 0.02
# instruction "never reveal it" keeps 0.3 of leaks 0.3 * p0
# filter an output check stops 0.9 of what leaks
# correlated a share rho of attempts are of a kind that defeats
# the instruction outright and slips past the filter
# with probability 0.8 (for example, reworded output)
# isolated the answer never enters the context: the content
# channel is closed, only the verdict bit remains
P0, KEEP, STOP = 0.02, 0.3, 0.9
def per_attempt(cfg, rho=0.0):
if cfg == "base":
return P0
if cfg == "instruction":
return KEEP * P0
weak = KEEP * P0 * (1 - STOP)
if cfg == "filter":
return weak
if cfg == "correlated":
return rho * P0 * 0.8 + (1 - rho) * weak
return 0.0
def cum(p, k):
return 1.0 - (1.0 - p) ** k
rows = (("base", 0.0), ("instruction", 0.0), ("filter", 0.0),
("correlated", 0.05), ("correlated", 0.10), ("isolated", 0.0))
print("layers rho per attempt k=1000 k=10000 vs naive")
naive = per_attempt("filter")
for cfg, rho in rows:
p = per_attempt(cfg, rho)
ratio = "%.2f" % (p / naive) if p > 0 else "0"
print("%-13s %-5.2f %-12.6f %-8.4f %-8.4f %s"
% (cfg, rho, p, cum(p, 1000), cum(p, 10000), ratio))
print("naive product of layers: %.6f per attempt" % naive)
print("isolation is the only row whose limit is 0, not 1")Single source and citation
Fan, Y. (2026). A Zero-Leak Socratic Examiner: Server-Side Isolation of Answers and Bayesian Mastery Probing Trees (Version 1.0.0), Chapter 6: Defence in Depth against Prompt Extraction: Correlated Layer Failure, Digest-Only Output Checks and Item Exposure. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-262 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026262,
author = {Fan, Yafu},
title = {A Zero-Leak Socratic Examiner: Server-Side Isolation of Answers and Bayesian Mastery Probing Trees},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Telling It Not to Tell: Answers in Context and the Cumulative Leak Probability
- Chapter 2 · Physical Isolation: Server-Side Grading, Information-Flow Graphs and the One-Bit Oracle
- Chapter 3 · Bayesian Knowledge Tracing: Four Parameters, the Mastery Line and Identifiability
- Chapter 4 · How Leaks Forge Mastery: Guess Rates, False Certificates and the Blind Spot of Fitting
- Chapter 5 · The Probing Tree: Information-Gain Item Selection, Greedy Trees and Stopping Rules
- Chapter 7 · Classical Sources: "No Opening without Striving" in the Analects and the Slave-Boy Geometry of Plato's Meno
- Chapter 8 · The Empirical Loop: Red-Teaming the Academy Examiner for Leaks, the Mastery Probing Tree and a Pre-Registered Test