Part Ⅺ · The arguments still live
Where the field disagrees.
ARC's progress has sparked recurring arguments about what its scores mean. This page puts five live debates side by side, with the evidence cited by each position and links to the relevant works in Part Ⅱ.
The 2024–2025 flashpoint
Is o3 AGI?
In December 2024, o3 cleared 87.5% on ARC-AGI-1 — the first human-competitive score — and set off a major AGI debate. Was that a genuine capability shift, or an expensive search that only resembles understanding?The two numbers aren’t equivalent: 75.7% came at high efficiency (~$20 a task); 87.5% only at a low-efficiency setting burning about 172× the compute — thousands of dollars a task, against roughly $5 for a human.
o3 does deep-learning-guided program search: it generates and evaluates candidate chains-of-thought at test time, so the reasoning trace is a task-specific program, not a retrieved answer. That is a new kind of test-time adaptability, not just a bigger model.
Passing ARC-AGI-1 is not AGI. o3 spends tens to thousands of dollars per task, still fails tasks humans find trivial, and early ARC-AGI-2 testing drops it under ~30% while humans stay above 95%. The score reflects brute breadth of search, not sample-efficient generalization.
Where it stands. Both sides agree on the facts and disagree on the word. o3 was a genuine advance in test-time adaptability, but not the cheap, sample-efficient generalization ARC was built to measure — which is why ARC-AGI-2 put cost on the scoreboard. By July 2026, o3 had been surpassed on ARC-AGI-1 and newer models also scored strongly on ARC-AGI-2; ARC-AGI-3 remained much farther from its human baseline. The debate now concerns efficiency, scaffolding, and transfer, not whether one 2024 model was AGI.
What is a solution?
What is “the rule” — a program, a prediction, or a compression?
Three families disagree on what it even means to solve a task: infer an explicit program, predict the output grid directly, or minimize the puzzle's description length.
The solution is an explicit, inspectable program (Python or a DSL) that maps input to output and can be verified against the demonstrations. Search the space of programs; keep what replays every pair.
No explicit program is needed. A neural network, adapted to the task's own demonstrations, predicts the output grid directly — and per-task test-time training makes this the strongest single lever of the 2024 season.
cites №08 · The test-time-training result that formalized per-task adaptation for ARC.
The correct rule is the one that most compresses the demonstrations. Solve by minimizing description length, with no pretraining and no external search — a principled objective the other two only approximate.
cites №16 · CompressARC's no-pretraining, pure-MDL approach.
Where it stands. Induction and transduction provably fail on disjoint sets of tasks and only reach human level when ensembled — a clue that no single representation is the whole answer, and an open question about whether MDL is the objective underneath both.
The fairness question
Are hard-coded priors cheating?
ARC is designed around Core Knowledge priors. Solvers differ sharply in how much of that structure they hand-code and how much they learn, which complicates comparison with the human baseline.
Humans have innate Core Knowledge, so giving a solver object, geometry, and counting primitives is fair game — it is matching the human prior, not cheating. The 2020 winner's ~142-primitive DSL held the frontier for three years on exactly this bet.
A hand-coded DSL is a prior the human baseline never got to design for itself; the fairer and more scalable path is to learn the primitives. DreamCoder grows its own library of reusable abstractions by compression, inventing the concepts rather than being handed them.
Where it stands. The benchmark calibrates against humans, so the question is which built-in priors are legitimate and which amount to an answer key. Learning a Core-Knowledge library rather than hand-coding it is both a stricter test and a harder research problem.
The frontier method
Search the space, or model the world?
ARC-AGI-3's 2025 preview rewarded informed search. Its first 2026 milestone adds vision-language policies, memory, structured actions, and executable tools. The live question is whether those systems merely explore better or induce reusable dynamics and goals that transfer.
Cheap coverage won the preview, and the milestone systems keep its lesson: use local models, short action queues, context eviction, and targeted heuristics so every interaction produces information. You do not need a perfect explicit simulator to win an early checkpoint.
cites №17 · The 2025 preview podium and the 2026 Milestone #1 architectures.
Search and local reflection can plateau when the state space explodes or a hidden goal requires relational understanding. The benchmark's four-capability decomposition still demands a predictive world model and discovered goal that transfer beyond one hidden evaluation.
cites №03 · The ARC-AGI-3 benchmark definition and its four required capabilities.
Where it stands. Exploration, memory, and tools win the current checkpoints. The stronger claim—transferable world-model synthesis plus goal inference—remains unproven, which is exactly the gap the benchmark exists to expose.
The definition problem
What should “solved” even mean?
o3 nearly saturated ARC-AGI-1 on accuracy alone. That exposed a deeper disagreement: is a benchmark solved when a system is accurate, when it is accurate and cheap, or only when it can't be gamed by training on the task space?
If a system reaches human-level accuracy on a novel, private test set, it has solved the benchmark — the score is the score, and o3's 87.5% is a real result regardless of price.
Intelligence is efficiency, so a benchmark isn't solved until it's solved cheaply. ARC-AGI-2 makes cost-per-task a first-class scored axis precisely so a system that pays thousands of dollars per task cannot claim victory.
Where it stands. Current evaluation practice increasingly considers both accuracy and cost, alongside contamination-resistant private sets and stable human baselines. Under that view, "solved" means efficient and general, not merely accurate.
Why the arguments matter
Disagreement is where the frontier is.
Each of these debates marks a place the field has not convergedUnderneath every argument on this page sits one scientific question: what is the formal account of the sample-efficient, out-of-distribution generalization ARC demands — a program-length / Solomonoff / MDL prior, a meta-learning story, or something about compositional structure in the hypothesis space? Nobody has it yet. — which is exactly where the next result will come from. Follow them into the unanswered questions.