Part Ⅻ · The research frontier
Where the frontier is still open.
Step back from any single paper and ask: which unanswered questions would actually move the field toward the fluid, sample-efficient, compositional generalization ARC was built to measure? A synthesis across the whole corpus — 8 north-star questions, then 9 themes, each linked to the works that frame it.
The shortlist
The highest-leverage questions.
These questions are ranked by expected leverage. The ★ marks questions about synthesizing a world-model program and accepting it only after it replays every recorded transitionARC-AGI-3's validator also builds a graph of reachable states and replays recorded win and loss traces to verify each environment..
-
1
★ Build a reusable world model from a few dozen actions.
At launch, calibrated humans solved every interactive environment while harness-free frontier AI scored below 1%. What learning procedure can fit an accurate, reusable model of the dynamics from only a few dozen moves — under a regime that caps actions at 5× the human baseline and rewards first-contact efficiency? This is the ARC-AGI-3 crux.
-
2
★ Infer the goal with no instructions.
In ARC-AGI-3 there is no stated objective and no reward until the win frame. Goal induction from pure interaction means proposing and revising hypotheses about which state is desirable from sparse, delayed feedback. At launch, this was one of the clearest gaps between calibrated humans and harness-free frontier AI.
-
3
★ Can an exactly-replaying program generalize to unseen states?
A world-model program that replays every recorded transition may still be wrong on unseen states. Does exact-replay acceptance predict held-out generalization, or only consistency with the observed history?
-
4
What replaces the execution oracle for the refinement loop?
The 2025 refinement loop depended on an exact-match-on-demonstration-pairs oracle that ARC-AGI-3 removes, because the goal is hidden and must be probed. What feedback signal makes propose→verify→iterate converge when there is nothing to check a candidate against?
-
5
Why does test-time adaptation beat in-context learning — and can it be cheap?
Fitting a few gradient steps to a task's demonstrations reliably outperforms feeding the same demonstrations in-context, a ~6× effect with no satisfying theory. Can it be made cheap enough to run inside a real-time interactive agent loop?
-
6
Is compression / MDL the objective the others approximate?
If lower-bit compression really correlates with correctness, MDL would unify induction and transduction and give the field a principled target. Why, then, does pure compression cap at ~20–34% — a capacity, optimization, or prior-mismatch limit?
-
7
Can the DSL of primitives be discovered, not hand-built?
Every hand-built DSL supplies substantial prior knowledge. Can a system learn the primitive set—for static grids and interactive dynamics—rather than relying on a fixed, human-authored grammar?
-
8
How much of the gap is perception vs reasoning vs exploration?
Some ARC failure is the model mis-parsing the grid, not failing to reason; some of the ARC-AGI-3 gap is failing to perceive state changes, not failing to plan. Without a decomposition that isolates each, we cannot tell which subproblem to attack first.
The nine themes
Every facet of one gap.
Each theme attacks the same efficiency gap: machines can reach some static-task scores, but often with far more data and compute than humans. Every theme links to the works that frame it.
Induction vs transduction vs compression
Three families disagree on how to represent a solution: infer an explicit program, predict the grid directly, or minimize description length. They fail on disjoint tasks and only reach human level ensembled — a clue, not an answer. Is MDL the objective the other two approximate?
Test-time adaptation: the dominant lever
Per-task fine-tuning was the story of 2024, and no one has a satisfying account of why it beats in-context learning on the same examples — nor whether it can be made cheap enough for an interactive, real-time agent that must adapt within a single episode without forgetting.
see №08
Search and the refinement loop
Sample a candidate, run it, observe the error, revise — the refinement loop drove 2025. But it needs a feedback signal: an execution oracle on the demonstration pairs. ARC-AGI-3 removes that oracle, so what drives the loop when the goal must be probed?
From static puzzles to interactive worlds
ARC-AGI-3 changes the task: explore, model dynamics, infer the goal, and plan without instructions, scored on action efficiency. Calibrated humans solve every included environment; launch models score below 1%.
see №03
World models as programs, verified by replay
Should a world model be a synthesized program accepted only when it exactly replays every recorded transition? It is a strong, verifiable stance — and the benchmark's own validator independently reinvents graph-of-states and faithful action-trace replay to build the environments.
Priors and Core Knowledge: build in vs learn
ARC controls for priors, yet solvers differ wildly in how much Core Knowledge they hand-code (Icecuber's 142 primitives) versus learn. How much innate prior is necessary, how much is cheating relative to the human baseline, and can the priors themselves be learned?
Perception vs reasoning: which bottleneck?
Some ARC failure is perceptual — the model mis-parses the grid — not a failure of reasoning. Does serializing grids as tokens versus a 2D object-centric encoder change what the system can reason about, and how much of the ARC-AGI-3 gap is perception rather than planning?
see №19
Evaluation, cost, contamination, and “solved”
ARC-AGI-2 made cost-per-task first-class; the reports warn of knowledge-dependent overfitting; ARC-AGI-3 scores action-efficiency against humans. Is cost the right second axis, are private rotating sets enough against contamination, and are the human baselines fair and stable?
see №02
A theory of the generalization ARC demands
Underneath the engineering sits the deepest question: what is the formal account of the sample-efficient, compositional, out-of-distribution generalization ARC requires? A program-length / Solomonoff prior, a meta-learning story, or something about compositional structure — and why can humans do it from so little?
see №01
Research Experiment Board
Turn a question into a result that could be wrong.
Each row names an intervention, a measure, a falsifier, and the artifacts needed to audit the result. That last column is part of the experiment—not release-day paperwork.
The deepest question
What is the generalization ARC demands?
Underneath the engineering is one unanswered question: what form of generalization does ARC require?
Candidate answers include a program-length or MDL prior, meta-learning, and compositional structure in the hypothesis space. Theory names the axis; the benchmark chapters and Methods show how systems approach it; History records their progress; Debates asks what the scores mean. This page records what remains unresolved.
The frontier is not a wall; it is a list. These are the questions whose answers would actually move us toward what ARC was built to measure.
→ The interactive frontier, in Part Ⅶ · ARC-AGI-3