5D Library

Chapter 8 Practice and a Pre-Registered Trial

5D Library › The 85% Rule for Optimal Learning, Myelination Kinetics and Deliberate Practice · 中文

Chapter 8 Practice and a Pre-Registered Trial

Note

The checklist in this chapter is for general learners and teachers; it is not a tool for diagnosis or treatment. If learning difficulties persist, markedly affect daily life or come with emotional distress, seek a professional assessment rather than only changing how you practise. The “80–85 %” in the checklist is a reference band, not a standard to be hit exactly.

Part 1 A four-step deliberate-practice checklist

Keeping accuracy within the reference band

The derivation in Chapter 1 assumes that the trainer knows the learner’s current level. In reality we do not, and can adjust only on the basis of recent successes and errors. Psychophysics has a well-developed family of methods for this, staircases: make it harder after a correct answer, easier after an error. Levitt’s “3-down-1-up” rule (harder after three correct answers in a row, easier after any error) settles at about 79.4 % correct (Levitt 1971); Kaernbach’s weighted up-down method is more flexible: one step harder after a correct answer, p/(1−p)p/(1-p) steps easier after an error, and it settles near accuracy pp (Kaernbach 1991). To aim at 85 %, the step down after an error should be about 5.67 times the step up after a correct answer.

A simulated learner answers four-option items (25 % chance of guessing right), and its skill improves slowly with practice. Compare the weighted up-down method (target 85 %), the same rule with a learner who does not improve, “3-down-1-up”, and a fixed set of easy items on which the learner scores about 95 % at the start.

Calculation

# Staircases for a simulated learner (4-option items, P = 0.25 + 0.75 Phi(theta - x), skill
# +0.01/trial). Weighted up-down (Kaernbach 1991): easier by s p/(1-p) after an error, harder
# by s after a correct answer; 3-down-1-up (Levitt 1971); fixed easy items (~95 % at start).
import random
from statistics import NormalDist
N = NormalDist()

def p_correct(theta, x):
    return 0.25 + 0.75 * N.cdf(theta - x)

def run(rule, seed, trials=300, target=0.85, step=0.08, gain=0.01):
    rng = random.Random(seed)
    theta, streak, hits = 0.0, 0, []
    x = theta - N.inv_cdf((0.95 - 0.25) / 0.75)               # start where accuracy = 95 %
    x0 = x
    for _ in range(trials):
        ok = rng.random() < p_correct(theta, x)
        hits.append(ok)
        if rule == "weighted":
            x += step if ok else -step * target / (1 - target)
        elif rule == "3down1up":
            streak = streak + 1 if ok else 0
            if streak == 3:
                x, streak = x + step * 2, 0
            elif not ok:
                x -= step * 2
        else:
            x = x0
        theta += gain
    return sum(hits[100:]) / 200, theta - x

print("500 simulated learners per rule | accuracy, trials 101-300 | final skill - difficulty")
for rule, label, gain in (("weighted", "weighted up-down, 85 %", 0.01),
                          ("weighted", "  same, learner not improving", 0.0),
                          ("3down1up", "3-down-1-up", 0.01), ("fixed", "fixed easy items", 0.01)):
    r = [run(rule, s, gain=gain) for s in range(500)]
    acc = sum(a for a, _ in r) / len(r)
    gap = sum(g for _, g in r) / len(r)
    print(f"{label:30s} | {acc:24.1%} | {gap:+.2f}")
# 500 simulated learners per rule | accuracy, trials 101-300 | final skill - difficulty
# weighted up-down, 85 %         |                    86.9% | +1.00
#   same, learner not improving  |                    85.0% | +0.91
# 3-down-1-up                    |                    83.3% | +0.79
# fixed easy items               |                    99.9% | +4.50

With a learner who does not improve, the weighted up-down method settles at 85.0 %; with a learner who is improving, the staircase always lags slightly behind, and actual accuracy is about 86.9 %. “3-down-1-up” gives about 83.3 % here (the theoretical value is 79.4 %, also raised because the learner is improving). Fixed easy items reach almost 100 % after 300 trials: the learner has long since outgrown their difficulty, exactly the “too good” of Chapter 1.

The checklist

The checklist is meant for a learner or teacher to keep for a week and then review. It applies to practice with clear right and wrong answers and graded difficulty (problem solving, vocabulary, applying rules), not to all learning.

  1. Is current accuracy within the reference band? Is accuracy in each session roughly 80–85 %? Accuracy that stays above about 90 % suggests the items may be too easy; accuracy that stays below about 75 % suggests they may be too hard, or that more basic material should come first. (Chapter 1)
  2. The ratio of active retrieval to passive review. How much time goes on answering, recalling and solving without notes, and how much on rereading, watching explanations and looking at answers? (Chapter 4)
  3. Interleaving of item types. Within a session, are different kinds of item mixed together, or is one kind finished before the next begins? For material that requires discriminating “which method applies”, interleaving helps more. (Chapter 5)
  4. Feedback delay and the correction loop. How soon after an item does the learner know whether it was right? Are wrong items passed over once the answer has been seen, or is the cause of the error understood and an item of the same kind then done again correctly? (Chapter 6)

The checklist applied to a week of practice records kept by a fictional learner: six sessions on four types of statistics problem (A–D).

Calculation

# Four-step checklist on a FICTIONAL week: item types, correct answers, active / passive
# minutes, median feedback delay (s), errors corrected and re-attempted.
LOG = [
    ("Mon", "AAAAAAAABBBBBBBB", "1111111111111111", 10, 35, 2, 0),
    ("Tue", "BBBBBBBBCCCCCCCC", "1111111011111111", 12, 30, 2, 0),
    ("Wed", "ABACBDCADBCADBAC", "1011011101110110", 30, 15, 5, 3),
    ("Thu", "CADBBCADACDBDABC", "1101110111011111", 32, 12, 5, 3),
    ("Fri", "DDDDDDDDDDDDDDDD", "0100010001000100", 25, 10, 120, 1),
    ("Sat", "BDACADBCCBDADCAB", "1110111111011101", 35, 10, 4, 3),
]
BAND = (0.75, 0.90)            # working band around the 80-85 % aim (generous on purpose)

def switch_rate(seq):
    return sum(a != b for a, b in zip(seq, seq[1:])) / (len(seq) - 1)

print("session | acc   step 1        | retrieval:passive | switches | fb delay | errors fixed")
tot_err = tot_fix = 0
for day, seq, ok, act, pas, delay, fixed in LOG:
    acc = ok.count("1") / len(ok)
    errs = ok.count("0")
    tot_err += errs; tot_fix += fixed
    s1 = "too easy" if acc > BAND[1] else "too hard" if acc < BAND[0] else "in band"
    print(f"{day:7s} | {acc:4.0%}  {s1:12s} | {act / pas:6.1f} : 1        |"
          f" {switch_rate(seq):7.0%}  | {delay:5d} s  | {fixed} of {errs}")
mix = [switch_rate(s) for _, s, *_ in LOG]
print(f"random order of 4 types switches about 75 % of the time; this week: "
      f"{sum(m > 0.6 for m in mix)} of 6 sessions interleaved")
print(f"all week: {tot_fix} of {tot_err} errors corrected and re-attempted"
      f" ({tot_fix / tot_err:.0%})")
# session | acc   step 1        | retrieval:passive | switches | fb delay | errors fixed
# Mon     | 100%  too easy     |    0.3 : 1        |      7%  |     2 s  | 0 of 0
# Tue     |  94%  too easy     |    0.4 : 1        |      7%  |     2 s  | 0 of 1
# Wed     |  69%  too hard     |    2.0 : 1        |    100%  |     5 s  | 3 of 5
# Thu     |  81%  in band      |    2.7 : 1        |     93%  |     5 s  | 3 of 3
# Fri     |  25%  too hard     |    2.5 : 1        |      0%  |   120 s  | 1 of 12
# Sat     |  81%  in band      |    3.5 : 1        |     93%  |     4 s  | 3 of 3
# random order of 4 types switches about 75 % of the time; this week: 3 of 6 sessions interleaved
# all week: 10 of 24 errors corrected and re-attempted (42%)

The fictional record shows the following. Monday and Tuesday are entirely blocked, at 100 % and 94 % accuracy, with about 2.5 to 3.5 times as much time rereading and watching explanations as actively solving: smooth practice that may teach little. On Wednesday, the first interleaved session, accuracy drops to 69 %; on Thursday and Saturday it holds at 81 %, with retrieval-to-review ratios of about 2.7 to 1 and 3.5 to 1. On Friday, blocked practice on a new type runs at only 25 %, with a two-minute wait for answers and only 1 of 12 errors corrected: difficulty should be lowered first. Over the week, 10 of 24 errors were corrected and redone (42 %). A week’s record can only suggest adjustments; it is not evidence.

Part 2 A pre-registered randomised controlled trial

Question

For adults learning a wholly new set of symbolic reasoning rules, does “adaptive difficulty holding practice accuracy at about 85 %, plus interleaved retrieval practice”, compared with “accuracy held at about 95 %, plus practice blocked by rule”, with practice time and number of items equal, improve scores on a transfer test 14 days later?

Learning material: an artificial rule system

To exclude the influence of prior knowledge, the material is a purpose-built artificial rule system. It consists of 8 meaningless graphic symbols; learners must master 4 rules that turn one string of symbols into another:

  • Mirror: reverse the order of the whole string;
  • Paired substitution: replace each symbol by its fixed “partner” (the 8 symbols form four pairs);
  • Rotation: move the first symbol to the end;
  • Conditional deletion: delete every symbol that immediately follows one particular symbol.

Each practice item gives a string and a rule marker and asks for the result, or gives a string before and after transformation and asks which rule was used. Difficulty is graded along three dimensions: string length (3 to 9 symbols), the number of rules applied in succession (1 to 3), and how similar the distractors are to the correct answer. The full design of rules and symbols, the item bank and the difficulty grading are published before enrolment.

Design

Element Specification
Participants Adults aged 18–65, fluent in the study language, with normal or corrected vision; recruited through university and community advertising, not from the author’s community, students, customers or card users; no diagnosis is used as an inclusion or exclusion criterion
Exclusions Previous participation in a study using the same rule system; unable to complete the 14-day follow-up
Randomisation Individual 1:1 randomisation, stratified by baseline matrix-reasoning score (above or below the median) and site; computer-generated with allocation concealed
Practice schedule Both arms practise on days 1, 3, 5 and 8, 100 items per session with a 35-minute limit; the correct answer is shown immediately after each response; number of items and time are the same in both arms
Arm A Difficulty adjusted by the weighted up-down method, target accuracy 85 %; items for the four rules fully interleaved
Arm B (active control) The same weighted up-down method with target accuracy 95 %; each session divided by rule into four blocks of 25 items, practised blocked
Blinding Participants are told only that “two practice programs” are being compared, not the study hypothesis; tests are scored automatically; data analysts do not know the allocation until unblinding
Primary outcome Accuracy on a transfer test on day 22 (14 days after the last practice session): 40 items, including rule combinations not seen in practice, the same rules with a new symbol set, and inverse items inferring the original string from the result
Secondary outcomes Retention test of practised item types on day 22; actual accuracy during practice (manipulation check); judgements of learning and frustration ratings after each session; attrition
Analysis Analysis of covariance: arm as factor, baseline matrix-reasoning score and site as covariates; two-sided α=0.05\alpha = 0.05; main analysis on all randomised participants with primary-outcome data, with multiple imputation as a sensitivity analysis

Sample size

Calculation

# Sample size, two-arm trial (Chapter 8): A = ~85 % adaptive + interleaved; B = ~95 % +
# blocked. Day-14 transfer test, ANCOVA with a baseline covariate correlating rho
# (residual variance 1 - rho^2). Normal approximation, two-sided alpha 0.05.
import math
from statistics import NormalDist
N = NormalDist()

def per_arm(d, rho=0.4, alpha=0.05, power=0.90):
    z = N.inv_cdf(1 - alpha / 2) + N.inv_cdf(power)
    return math.ceil(2 * z * z * (1 - rho * rho) / (d * d)) + 1

print("analysable participants per arm, 90 % power")
print(" effect d | covariate rho = 0.0 |  0.4 |  0.6")
for d in (0.25, 0.30, 0.35, 0.40):
    print(f"   {d:.2f}   | {per_arm(d, 0.0):19d} | {per_arm(d, 0.4):4d} | {per_arm(d, 0.6):4d}")
n = per_arm(0.30, 0.4)
rec = math.ceil(n / 0.85)
print(f"chosen: d = 0.30, rho = 0.4 -> {n} per arm")
print(f"   allowing 15 % loss by day 14: recruit {rec} per arm, {2 * rec} in all")
print("optional 2 x 2 factorial (difficulty 85 % vs 95 %) x (interleaved vs blocked):")
print(f"   each main effect at d = 0.30: {4 * math.ceil(n / 2)} analysable"
      f" ({math.ceil(n / 2)} per cell), if the factors do not interact")
ni = per_arm(0.30 / 2, 0.4)
print(f"   an interaction of 0.30 (difference of differences): about"
      f" {4 * math.ceil(ni / 2)} analysable ({math.ceil(ni / 2)} per cell)")
# analysable participants per arm, 90 % power
#  effect d | covariate rho = 0.0 |  0.4 |  0.6
#    0.25   |                 338 |  284 |  217
#    0.30   |                 235 |  198 |  151
#    0.35   |                 173 |  146 |  111
#    0.40   |                 133 |  112 |   86
# chosen: d = 0.30, rho = 0.4 -> 198 per arm
#    allowing 15 % loss by day 14: recruit 233 per arm, 466 in all
# optional 2 x 2 factorial (difficulty 85 % vs 95 %) x (interleaved vs blocked):
#    each main effect at d = 0.30: 396 analysable (99 per cell), if the factors do not interact
#    an interaction of 0.30 (difference of differences): about 1572 analysable (393 per cell)

With a smallest effect of interest of 0.30 standard deviations, a correlation of 0.4 between the baseline covariate and the outcome, and 90 % power, each arm needs 198 analysable participants; allowing for about 15 % loss over the 14-day follow-up, 233 are recruited per arm, 466 in all. The higher the covariate correlation, the smaller the sample needed; the table gives the figures for correlations of 0 and 0.6.

A combined intervention

Arms A and B differ in target accuracy (85 % against 95 %) and in practice order (interleaved against blocked), so even a positive result cannot show which did the work. To separate them, a 2 × 2 factorial design could be used. The program shows that, if the factors do not interact, the same 396 analysable participants (99 per cell) test each main effect with the same power, whereas an interaction of the same size (0.30 standard deviations) needs about 1,572 (393 per cell). The protocol uses two arms because it first answers a practical question: is this combination better than the common “easy items, practised chapter by chapter”?

Decision rule

Calculation

# Decision rule fixed in advance. d = A minus B (ANCOVA, pooled SD); positive favours A.
# Manipulation check: A's median practice accuracy 78-90 %, B's >= 92 %. SESOI 0.30 SD.
SESOI = 0.30

def verdict(d, lo, hi, acc_a, acc_b, drop_gap):
    if not (0.78 <= acc_a <= 0.90 and acc_b >= 0.92):
        return (f"d = {d:+.2f}: NOT INTERPRETABLE - practice accuracy A {acc_a:.0%},"
                f" B {acc_b:.0%} missed the planned contrast")
    note = ""
    if drop_gap > 0.10:
        note = ("\n   + arm A lost over 10 percentage points more participants: report as"
                " a possible burden signal")
    if lo >= SESOI:
        v = "arm A improves transfer by at least the smallest effect of interest"
    elif lo > 0:
        v = "arm A improves transfer; size not shown to reach 0.30 SD"
    elif hi < 0:
        v = "arm A WORSENS transfer"
    elif lo >= -SESOI and hi <= SESOI:
        v = "no difference worth acting on"
    else:
        v = "unclear: the interval is too wide to decide"
    return f"d = {d:+.2f} ({lo:+.2f} to {hi:+.2f}): {v}{note}"

for case in ((0.48, 0.32, 0.64, 0.84, 0.95, 0.03), (0.20, 0.04, 0.36, 0.85, 0.96, 0.02),
             (0.05, -0.11, 0.21, 0.83, 0.95, 0.01), (0.10, -0.25, 0.45, 0.86, 0.94, 0.04),
             (0.40, 0.24, 0.56, 0.95, 0.96, 0.02), (0.35, 0.19, 0.51, 0.82, 0.95, 0.14)):
    print(verdict(*case))
print("secondary outcomes (retention of trained rules, frustration, judgments of learning)"
      "\n   are reported with intervals and never decide the verdict")
# d = +0.48 (+0.32 to +0.64): arm A improves transfer by at least the smallest effect of interest
# d = +0.20 (+0.04 to +0.36): arm A improves transfer; size not shown to reach 0.30 SD
# d = +0.05 (-0.11 to +0.21): no difference worth acting on
# d = +0.10 (-0.25 to +0.45): unclear: the interval is too wide to decide
# d = +0.40: NOT INTERPRETABLE - practice accuracy A 95%, B 96% missed the planned contrast
# d = +0.35 (+0.19 to +0.51): arm A improves transfer; size not shown to reach 0.30 SD
#    + arm A lost over 10 percentage points more participants: report as a possible burden signal
# secondary outcomes (retention of trained rules, frustration, judgments of learning)
#    are reported with intervals and never decide the verdict

The example results are made up. The manipulation check is a precondition of any verdict: if arm A’s median practice accuracy is not between 78 % and 90 %, or arm B’s is below 92 %, the trial has not tested the contrast it set out to test, and the report says “not interpretable” whatever the result. Only if the lower limit of the effect’s confidence interval is at or above 0.30 standard deviations does the report say “arm A improved transfer by at least the smallest effect of interest”. If arm A loses more than 10 percentage points more participants than arm B, this is reported as a possible burden signal even if the primary outcome is positive.

What this trial cannot show

First, it is a combined intervention and cannot separate the contributions of the 85 % and of interleaving (see above). Second, the material is an artificial rule system; whether the results extend to real subjects such as mathematics, languages or medical diagnosis needs further research. The artificial system excludes prior knowledge, at the cost of distance from real learning. Third, the primary outcome is measured after 14 days and cannot show retention over months or years. Fourth, practice in arm A is harder and may be more frustrating; the trial records frustration and attrition, but cannot show whether learners would persist with such practice in everyday learning without study payments and a research setting. Fifth, this is not a study of learning disabilities or any clinical condition, and its results cannot be used for diagnosis or treatment.

Ethics and conflicts of interest

The trial must be approved by an independent ethics committee, participants must give informed consent, and the protocol, item bank and analysis plan must be registered before the first participant is enrolled (Nosek et al. 2018). Participants may withdraw at any time without giving a reason and are paid for the parts they complete. Whether the result is positive, negative or not interpretable, the results, data and analysis code will be published.

Disclosure

The author uses the works of 5D Library, including this book, as the methodological basis of his community courses, the 5D Ascension Cards (Dimensional Shift Card Decks) and the Yafu AI knowledge base, and therefore has a commercial interest in the subject of this book. The book promotes no course, card deck or service; the Chapter 8 trial is designed to be run and analysed by researchers independent of the author’s business and does not recruit from the author’s community, students or customers; all the code and the study protocol are published openly for anyone to check. To limit the influence of this interest: the item bank, practice program and analysis code are published before enrolment; whatever the result, the results, data and code will be published.

References

Kaernbach, Christian. 1991. “Simple Adaptive Testing with the Weighted up-down Method.” Perception & Psychophysics 49 (3): 227–29. https://doi.org/10.3758/BF03214307.
Levitt, H. 1971. “Transformed up-down Methods in Psychoacoustics.” The Journal of the Acoustical Society of America 49 (2B): 467–77. https://doi.org/10.1121/1.1912375.
Nosek, Brian A., Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor. 2018. “The Preregistration Revolution.” Proceedings of the National Academy of Sciences 115 (11): 2600–2606. https://doi.org/10.1073/pnas.1708274114.

Fan, Y. (2026). The 85% Rule for Optimal Learning, Myelination Kinetics and Deliberate Practice: From Stochastic Gradient Descent and the Zone of Proximal Development to Spaced Repetition, Read Beside Classical Chinese Learning Theory [学习的85%最优错误率定律、髓鞘化动力学与刻意练习] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23761339 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).