Is more feedback always better?

Almost all advice about practice stresses feedback: knowing whether you were right, and where you went wrong. That is broadly sound, but the picture research gives is far more complicated than “the more and the sooner the better”. Feedback can support learning, and it can replace it; it can direct attention to the task, and it can direct attention to “am I stupid?”.

The guidance hypothesis

What an average effect hides

Two parts. Part 1: if the effect sizes of feedback interventions were normally distributed (an illustrative assumption) with a mean of 0.41 and a third below zero, how wide would the distribution be? Part 2: a toy model of the guidance hypothesis. A learner practises a 500 ms timing movement, starting with a bias of +120 ms; with external feedback (KR) it corrects 30 % of the observed error; without KR it must rely on its own estimate of the error, whose noise shrinks with the number of trials on which it has “really attended to its own feedback”. The key parameter is “engagement”: the probability that, on a trial with KR, the learner still makes its own estimate first. A model constructed by the author, with illustrative parameters.

# Part 1: Kluger & DeNisi (1996) mean d = 0.41, a third negative: implied spread if normal.
# Part 2 (author's toy model): 500 ms timing task, bias +120, motor SD 40; KR corrects 30 % of
# error; own estimate noise 120/sqrt(1 + n/8), n = trials attending to own feedback (always
# without KR, with prob. "engage" with KR). Retention: drift SD 30, own-estimate correction.
import math, random
from statistics import NormalDist, fmean

mu, frac = 0.41, 1 / 3
sd = mu / -NormalDist().inv_cdf(frac)
D = NormalDist(mu, sd)
print(f"Part 1: mean d = {mu}, one third below zero -> SD of effects about {sd:.2f}")
print(f"   share below -0.2: {D.cdf(-0.2):.0%}; between 0 and 0.4: {D.cdf(0.4) - D.cdf(0):.0%};"
      f" above 0.8: {1 - D.cdf(0.8):.0%}")

def schedule(name, n=120):
    if name == "100 %":
        return [True] * n
    if name == "50 %":
        return [i % 2 == 0 for i in range(n)]
    return [random.Random(i).random() < max(0.0, 1 - i / (0.8 * n)) for i in range(n)]

def learner(sched, engage, seed):
    rng = random.Random(seed)
    bias, n_att, acq = 120.0, 0, []
    for kr in sched:
        err = bias + rng.gauss(0, 40)
        acq.append(abs(err))
        own = err + rng.gauss(0, 120 / math.sqrt(1 + n_att / 8))
        if kr:
            bias -= 0.3 * err
            n_att += rng.random() < engage
        else:
            bias -= 0.3 * own
            n_att += 1
    bias += rng.gauss(0, 30)
    ret = []
    for _ in range(30):
        err = bias + rng.gauss(0, 40)
        ret.append(abs(err))
        bias -= 0.3 * (err + rng.gauss(0, 120 / math.sqrt(1 + n_att / 8)))
    return fmean(acq[-40:]), fmean(ret)

print("Part 2: mean absolute error (ms), last 40 practice trials / one-day retention test")
print("KR schedule | engage = 0.1 (strong guidance) | engage = 0.9 (learner self-estimates)")
for name in ("100 %", "50 %", "faded"):
    cells = []
    for engage in (0.1, 0.9):
        r = [learner(schedule(name), engage, s) for s in range(2000)]
        cells.append(f"{fmean(a for a, _ in r):5.1f} / {fmean(b for _, b in r):5.1f}")
    print(f"{name:11s} |        {cells[0]}         |          {cells[1]}")
# Part 1: mean d = 0.41, one third below zero -> SD of effects about 0.95
#    share below -0.2: 26%; between 0 and 0.4: 16%; above 0.8: 34%
# Part 2: mean absolute error (ms), last 40 practice trials / one-day retention test
# KR schedule | engage = 0.1 (strong guidance) | engage = 0.9 (learner self-estimates)
# 100 %       |         34.7 /  43.2         |           34.7 /  36.6
# 50 %        |         36.1 /  37.7         |           35.5 /  36.8
# faded       |         37.3 /  37.6         |           36.2 /  36.9

The arithmetic of Part 1 shows that “an average effect of 0.41” and “a third of interventions are harmful” do not contradict each other: under this normal assumption the standard deviation of the effects is about 0.95, about 26 % of interventions have effects below −0.2, and at the same time about 34 % are above 0.8. Behind the average lies a very wide distribution; the question is not “feedback or not” but “what kind of feedback”.

The model in Part 2 shows the logic of the guidance hypothesis, and also how conditional it is. When the learner hardly makes its own estimate on KR trials (engagement 0.1), feedback on every trial is best during practice (mean error over the last 40 trials 34.7 ms) but worst on the retention test a day later (43.2 ms); feedback reduced to 50 % or faded is slightly worse in practice and much better at retention (about 37.6–37.7 ms). But when the learner habitually makes its own estimate even on KR trials (engagement 0.9), 100 % feedback does just as well at retention (36.6 ms) and reduced feedback has no advantage. In this model, then, what matters is not the frequency of feedback as such, but whether the learner processes its own error information. This may be one reason why experiments on reduced feedback frequency disagree; but that is the author’s conjecture and needs experimental testing.

When to correct at once, and when to wait

Putting this research together with Chapters 4 and 5, the author’s synthesis (not the conclusion of any single study) is as follows. Immediate correction is favoured where an error could cause harm (laboratory procedures, driving, equipment); where the same error would be repeated many times before feedback arrives (a whole problem set done with the wrong formula); and at the beginning, when learners have no internal standard yet. Metcalfe’s review notes that errors made during learning are generally not “fixed in” provided they are subsequently corrected, and that high-confidence errors are if anything more readily corrected; the proviso is that there is correction (Metcalfe 2017). Delayed or reduced feedback is worth considering when learners already have some capacity for self-assessment and must detect errors without external feedback; after retrieval practice, recalling first and seeing the answer afterwards (Chapter 4); and when the learner’s metacognitive judgement should be preserved, for example by giving a confidence rating before feedback. In every case feedback should point to the task and the process, not to the person.

Chapter summary

The guidance hypothesis holds that external feedback on every trial improves performance in practice but may weaken the learner’s processing of its own errors; Winstein and Schmidt found that reduced feedback frequency helped retention, but a later meta-analysis did not give stable support. Kluger and DeNisi found an average effect of feedback interventions of d = 0.41, with more than a third making performance worse; feedback directed at the self is especially likely to harm. The model suggests that the key may be not the frequency of feedback but whether the learner processes its own error information. Immediate correction suits safety, errors that would be repeated many times, and the beginning; delayed or reduced feedback suits the stage at which self-assessment is to be developed. The next chapter reads this modern research beside classical Chinese thought on learning.