5D Library › The Algorithmic Mirror · 中文
Chapter 6 Centaur Intelligence
From freestyle chess
After losing to IBM’s Deep Blue in 1997, Garry Kasparov began promoting “advanced chess”, in which each player works with a computer. In 2005 the online PAL/CSS Freestyle tournament, open to any combination of people and machines, was won not by a grandmaster with a supercomputer but by two amateurs from New Hampshire, Steven Cramton and Zackary Stephen, rated about 1700 and 1400, using three ordinary computers and several engines (ChessBase 2005). Kasparov later drew the lesson that a weak human with a machine and a better process could beat both a strong computer alone and a strong human with a machine and a worse process (Kasparov 2010). The image of the centaur, half human and half machine, entered the vocabulary of human–AI collaboration.
Note
Chess engines have since grown far stronger, and the margin a human partner can add at the top level has shrunk. The freestyle story is a vivid example of good process, not evidence that human–AI teams win in general.
What the evidence shows
Evidence
In a field experiment with 758 consultants at Boston Consulting Group, Dell’Acqua and colleagues found that on tasks inside the frontier of what GPT-4 could do well, consultants using it completed 12.2 % more tasks, 25.1 % faster, with more than 40 % higher quality; but on a task chosen to lie just outside that frontier, consultants using it were 19 percentage points less likely to reach a correct answer than those working without it. They described the boundary of AI capability as a jagged frontier, and observed two styles of successful use: “centaurs”, who divided tasks between themselves and the AI, and “cyborgs”, who interleaved their work with it closely (Dell’Acqua et al. 2023).
Evidence
A meta-analysis by Vaccaro, Almaatouq and Malone of 106 experiments (370 effect sizes) found that, on average, human–AI combinations performed worse than the better of the human alone or the AI alone (Hedges’ ), although better than humans alone (). Combinations lost most in decision tasks () and did relatively better in creative, open-ended tasks (, not significantly different from zero). When humans alone outperformed the AI alone, combinations beat both (); when the AI alone outperformed humans, combinations fell short of the AI () (Vaccaro et al. 2024).
So the claim that human–AI teams systematically beat both pure human experts and pure AI is not supported as a general rule. Teams win under particular conditions, and the evidence points to what those conditions are.
When does a team win?
Calculation
A jagged frontier in miniature: on 60 % of tasks the AI is right 90 % of the time, on the rest 40 %; the human is right 70 % of the time everywhere, so alone they are equal. In the team, the human decides when to trust the AI, recognising which kind of task it is with a given accuracy.
import random
# A "jagged frontier": on 60 % of tasks the AI is right 90 % of the time; on the rest, 40 %.
# The human alone is right 70 % of the time everywhere. In the team, the human decides
# whether to accept the AI's answer or use their own, and recognises which kind of task
# it is with some accuracy ("discernment").
random.seed(5)
def team(discernment, n=100000):
right = 0
for _ in range(n):
inside = random.random() < 0.6
ai_ok = random.random() < (0.9 if inside else 0.4)
human_ok = random.random() < 0.7
thinks_inside = inside if random.random() < discernment else not inside
right += ai_ok if thinks_inside else human_ok
return right / n
ai_alone = 0.6 * 0.9 + 0.4 * 0.4
print(f"AI alone {ai_alone:.2f}, human alone 0.70")
for d in (0.5, 0.7, 0.85, 1.0):
print(f"discernment {d:4.2f}: team accuracy {team(d):.3f}")
# AI alone 0.70, human alone 0.70
# discernment 0.50: team accuracy 0.697
# discernment 0.70: team accuracy 0.746
# discernment 0.85: team accuracy 0.784
# discernment 1.00: team accuracy 0.820
The team gains only through the human’s knowledge of where the AI is reliable and where it is not. That is a form of the requisite variety of Chapter 3: the human must be able to distinguish the cases.
The architect–verifier division of labour
The evidence suggests a division of labour that plays to the conditions under which teams win. The human defines the question and the standards of an answer, connects fields the model would not connect unprompted, judges which outputs can be checked against the world, and does the checking or arranges for it; the machine drafts, formalises, calculates, searches and writes code that can be run. This book was made that way. Whether this division beats the alternatives for a given task is an empirical question, and the next chapter’s classics and the last chapter’s trial are offered in that spirit.
Hypothesis · not established
Not established. In research-planning tasks, human–AI teams in which the human explicitly assesses, before accepting each output, whether it lies inside or outside the model’s reliable range outperform both the model alone and the human alone, whereas teams without that step do not. The hypothesis predicts a positive team-versus-best-alone difference only in the explicit-assessment arm of a three-arm randomised experiment. It would be refuted if the explicit-assessment teams did not outperform the better of human alone and model alone.
References
Fan, Y. (2026). The Algorithmic Mirror: Self-Attention Geometry, Ashby's Requisite Variety and Cognitive Projection in Information Theory and East Asian Philosophy of Mind [算法之镜] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23723557 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).