Directed state graph
Blind Squirrel / Just Explore
- Represent
- Observed states and transitions
- Explore
- Search unvisited graph edges
- Remember
- Explicit state graph
- Verify
- State equality and replay
- Plan
- Graph search
Evidence boundary: Limited
Appendix G · Visual index · 31 plates
The book's visual evidence, in one place.
Browse every chart, map, leaderboard, and data table in source-page order. Each plate matches its chapter appearance and links back to the surrounding argument.
31
Figures & tables
8
Source pages
15
Figures
16
Tables
The complete visual set
Read the field by its shapes.
Figures appear in source-page orderFigures and tables use separate counters, so a plate number identifies its kind, not its position in this gallery.. Use each source link to return to the argument and annotations around the visual.
Three generations of ARC
How task format, action, feedback, scoring, and failure mode change from ARC-AGI-1 to ARC-AGI-3.
Question answered: What changed when ARC moved from static transformations to instruction-free interactive worlds?
These are different evaluation regimes. Score percentages should not be compared as one continuous series.
| Dimension | ARC-AGI-12019–2024 | ARC-AGI-22025– | ARC-AGI-32025– |
|---|---|---|---|
| Task | Infer one transformation from a few examples. | Infer harder, more compositional transformations. | Discover and solve an unseen interactive game. |
| Observation | Static input/output grids. | Static input/output grids with fewer shortcuts. | A changing 64×64, 16-color world. |
| Agent action | Submit up to two output grids. | Submit up to two output grids. | Press a key, undo, or click a cell each turn. |
| Feedback | Demonstration pairs only. | Demonstration pairs only. | The next frame after every action; no instructions. |
| Scoring | Exact-match pass@2. | Exact-match pass@2 under a cost cap. | Completion plus Relative Human Action Efficiency. |
| Failure exposed | Few-shot rule induction. | Composition, refinement, and efficiency. | Exploration, world modeling, goal acquisition, and planning. |
The ARC-AGI timeline
Every catalogued work, placed by year and primary theme.
The web of ideas
How the papers build on, answer, and compete with one another.
The human-vs-AI gap
Best reported AI score against the human baseline for each benchmark.
ARC-AGI-12019
87.5%
98%
ARC-AGI-22025
24.03%
100%
ARC-AGI-32026
12.58%
100%
Best AI
Human baseline
▨ the gap that remains
The climb on ARC-AGI-1
The record’s rise from Icecuber to o3.
Cumulative works in the Atlas
How the selected catalog grows over time.
Influence × year
Every work by year, influence, and primary theme.
Works per year
Selected works by publication year.
Works by theme
The thematic composition of the Atlas.
Works by format
Papers, reports, posts, code, and other source formats.
Recurring authors
Names appearing on two or more works.
Most-connected works
Works with the most cross-references to other Works.
The chronological catalog
All works ordered by year with theme and influence.
#YearWorkThemeInfluence
01№01
2019
On the Measure of IntelligenceFrançois Chollet · paper
Benchmark
5
02№07
2020
ARC 2020 Kaggle 1st-Place Solution (Icecuber)Johan S. Wind · code
Milestone
5
03№18
2020
DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learningEllis et al. · paper
Foundations
4
04№14
2023
Hypothesis Search: Inductive Reasoning with Language ModelsWang et al. · paper
Synthesis
4
05№19
2023
Large Language Models as General Pattern MachinesMirchandani et al. · paper
Foundations
3
06№06
2024
OpenAI o3 Breakthrough High Score on ARC-AGI-PubARC Prize · blog
Milestone
5
07№04
2024
ARC Prize 2024: Technical ReportChollet et al. · report
Benchmark
4
08№08
2024
The Surprising Effectiveness of Test-Time Training for Abstract ReasoningAkyürek et al. · paper
Test-time
4
09№10
2024
Getting 50% (SoTA) on ARC-AGI with GPT-4oGreenblatt et al. · blog
Synthesis
4
10№11
2024
How I got a record 53.6% on ARC-AGI-Pub using Sonnet 3.5 with Evolutionary Test-time ComputeJeremy Berman · blog
Synthesis
4
11№12
2024
Combining Induction and Transduction for Abstract ReasoningLi et al. · paper
Synthesis
4
12№13
2024
Searching Latent Program Spaces (LPN)Bonnet et al. · paper
Neural
3
13№15
2024
CodeIt: Self-Improving Language Models with Prioritized Hindsight ReplayButt et al. · paper
Synthesis
3
14№02
2025
ARC-AGI-2: A New Challenge for Frontier AI Reasoning SystemsChollet et al. · paper
Benchmark
5
15№09
2025
Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of PerspectiveFranzen et al. · paper
Test-time
5
16№17
2025
Graph-Based Exploration for ARC-AGI-3 Interactive Reasoning TasksRudakov et al. · paper
Agentic
4
17№24
2025
Don't throw the baby out with the bathwater: How and why deep learning for ARCCole et al. · paper
Critique
4
18№16
2025
ARC-AGI Without Pretraining (CompressARC)Liao et al. · paper
Neural
3
19№23
2025
Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGIPfister et al. · paper
Critique
2
20№03
2026
ARC-AGI-3: A New Challenge for Frontier Agentic IntelligenceChollet et al. · paper
Agentic
5
21№22
2026
Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?Sergey Rodionov · paper
Agentic
5
22№05
2026
ARC Prize 2025: Technical ReportChollet et al. · report
Benchmark
4
23№20
2026
Executable World Models for ARC-AGI-3 in the Era of Coding AgentsSergey Rodionov · paper
Agentic
4
24№21
2026
OPINE-World: Programmatic World Modeling with Ontology-error-Prioritized Interactive Exploration for ARC-AGI-3Courtis et al. · paper
Agentic
4
The price of an extra action
How ARC-AGI-3’s efficiency term falls as an agent spends more actions than a human.
Question answered: How sharply does ARC-AGI-3 punish actions beyond the human baseline?
This curve includes the official 1.15 per-level cap. The per-game completion cap, level weighting, and benchmark aggregation are separate parts of the final score.
human actions → efficiency
level efficiency = min(1.15, (human actions / agent actions)²)
| Agent actions | 0.75× | 1× | 2× | 3× | 4× | 5× |
|---|---|---|---|---|---|---|
| Efficiency term | 115% | 100% | 25% | 11.1% | 6.25% | 4% |
Source: ARC-AGI-3 scoring methodology. Retrieved 2026-07-28.
ARC-AGI-3 preview leaderboard
Selected preview scores, with the harness-free frontier shown for comparison.
1
StochasticGoose · Tufa Labs
12.58%
2
Blind Squirrel
6.71%
3
Frontier LLMs · harness-free
<1%
Capabilities supported by published evidence
Published evidence for exploration, modeling, goal inference, planning, and harness-free generalization.
Question answered: Which parts of the ARC-AGI-3 loop does each reported solver actually demonstrate?
Statuses summarize published evidence, not an intrinsic capability score. “Generalize” is an added cross-cutting diagnostic, not one of the benchmark’s four core capabilities.
| System | Explore | Model | Infer goal | Plan | Generalize |
|---|---|---|---|---|---|
| The Duck1st · 2026 Milestone Prize #1 | YesMultimodal views, segmentation, executable Python, and a private milestone win show effective first-contact exploration. | PartialPython supports explicit local hypotheses, but no transferable transition model is reported. | PartialThe agent can reason about progress during play; the report does not isolate independent hidden-goal induction. | YesThe REPL loop supports multi-step reasoning, action, observation, and revision. | YesPlaced first on the hidden holdout for 2026 Milestone #1. |
| Reki2nd · 2026 Milestone Prize #1 | YesRare-color clicks, dead-signatures, recent-frame vision, and reflection reduce wasted exploration. | PartialStructured observations and reflection preserve local beliefs, but no reusable dynamics model is reported. | PartialThe policy writes a plan from visual evidence; the milestone report does not isolate goal induction. | YesShort action queues and recurring reflection support plan execution and revision. | YesPlaced second on the hidden holdout for 2026 Milestone #1. |
| forge3rd · 2026 Milestone Prize #1 | YesA local visual policy, structured actions, and persistent memory reached the milestone podium. | PartialThe configurable framework can support code generation, but the winning profile did not establish a transferable model. | PartialReflection can retain hypotheses about progress; independent goal induction remains unmeasured. | YesStructured action selection with repair and guards supports reliable local execution. | YesPlaced third on the hidden holdout for 2026 Milestone #1. |
| StochasticGoose12.58% · 1st, preview | YesLearned policy predicts state-changing actions and searches from them. | PartialPredicts useful action effects, but does not induce an explicit dynamics model. | UnprovenNo reported mechanism for inferring the hidden win condition. | PartialSearch supplies short-horizon action selection rather than goal-directed planning. | YesRanked first on the private preview without an environment-specific harness. |
| Blind Squirrel6.71% · 2nd, preview | YesBuilds and searches an observed state graph. | PartialStores observed transitions but does not infer the rule producing them. | UnprovenNo reported hidden-goal induction. | PartialUses graph search to reach unexplored states. | YesRanked second on the private preview without a hand-built game harness. |
| Just-Explore (graph-based explorer)3.64% · 3rd, preview · 12 private levels | YesFrontier-driven exploration is the system's central mechanism. | PartialRecords state-action transitions but never induces a transition program. | UnprovenNo goal-acquisition component is reported. | PartialPlans paths to the nearest state with an untested action. | YesTraining-free policy was evaluated on private preview levels. |
| Duke LRM harnessall 3 public envs, ~human action counts · no transfer | PartialLLM queries can choose actions, but exploration depends on a custom harness. | PartialHistory compression supports local reasoning without a reusable learned dynamics model. | PartialCan reason about goals on the seen public environments. | YesAchieved human-like action counts on all three public environments. | NoReported gains did not transfer to unseen private environments. |
| Arcgenticaall 3 public envs, ~human action counts · no transfer | PartialAgent orchestration supports action selection on seen environments. | PartialSubagents reason over traces, but no reusable transition model is reported. | PartialCan infer progress on the public games with harness support. | YesCompleted all three public environments at roughly human action counts. | NoThe environment-specific harness did not transfer to private games. |
| Frontier LLMs (harness-free)~0.1–0.5% · <1% at launch | PartialModels act, but stall after relatively few expensive model-gated steps. | UnprovenNo robust world-model induction is demonstrated by the sub-1% results. | UnprovenHidden-goal inference remains a reported failure mode. | PartialCan propose action sequences, but does not revise them reliably under uncertainty. | PartialThe same harness-free protocol applied across hidden environments, but launch-era performance remained below 1%. |
DemonstratedPartial evidenceNot yet demonstratedNot demonstrated
Sources: ARC-AGI-3 Technical Report · ARC-AGI-3 Preview: 30-Day Learnings · ARC Prize 2026: ARC-AGI-3 Milestone Prize #1.
The climb and the resets
The frontier score across all three ARC-AGI benchmarks.
ARC-AGI-1ARC-AGI-2ARC-AGI-3
Human ≈ 98–100%
ARC-AGI-1 selected results
Reported scores across public, private, and ARC-AGI-Pub splits.
1
OpenAI o3 · OpenAI
87.5%
2
MindsAI · Cole, Osman, Hodel
55.5%
3
Berman — Evolutionary · Jeremy Berman
53.6%
4
The ARChitects · Franzen, Disselhoff, Hartmann
53.5%
5
MIT — Test-Time Training · Akyürek et al. · MIT
53%
6
Greenblatt (GPT-4o) · Ryan Greenblatt · Redwood
42%
7
Icecuber · Johan S. Wind, solo
20%
ARC-AGI-2 selected results
Reported scores and per-task costs for the 2025 season's leading systems.
1
Gemini-3-Pro harness · application-layer refinement
54%
2
2025 Kaggle winner · top private entry
24.03%
3
Tiny Recursive Model (TRM) · weight-space refinement
8%
4
Frontier reasoning models · o3 · o1-pro · Claude 3.7
3%
Selected ARC-AGI-2 score–cost frontiers
Frontiers among selected systems, computed separately for each evaluation class.
Question answered: Which selected ARC-AGI-2 systems buy more score for their reported inference cost?
Frontier status is computed only within the same declared evaluation class and only among the systems shown. Contest, commercial-model, and refinement results share axes for orientation but are not treated as one controlled evaluation.
2025 contest · private evalARC Prize verified · commercial modelsARC Prize verified · refinement harness
Non-dominated within shown evaluation class
| System | Score | Cost/task | Protocol | Status |
|---|---|---|---|---|
| NVARC | 24.03% | $0.2 | 2025 Kaggle private evaluation; contest constraints | Frontier among shown peers |
| Gemini 3 Pro | 31% | $0.81 | ARC Prize verified commercial baseline | Frontier among shown peers |
| Opus 4.5 | 37.6% | $2.2 | Thinking, 64k; ARC Prize verified | Frontier among shown peers |
| Poetiq + Gemini 3 Pro | 54% | $30 | Open refinement harness; ARC Prize verified | Frontier among shown peers |
Snapshot 2025-12-05. Source: ARC Prize 2025 Results & Analysis.
ARC-AGI-3 selected results
Preview and harness-free systems shown with their evaluation context.
1
StochasticGoose · Tufa Labs
12.58%
2
Blind Squirrel
6.71%
3
Just-Explore · Rudakov, Shock & Cowley
3.64%
4
Frontier LLMs (harness-free) · major labs
<1%
Method genealogy: recurring mechanisms
A reading map of combinations and methodological analogies across ARC solver families.
Question answered: Which mechanisms recur or combine across ARC solver families?
A curated reading map of recurring mechanisms. Solid edges mark a combination; dashed edges mark a close methodological analogy. Neither establishes citation volume or causal descent.
DSL search2020 · Synthesis
DreamCoder2020 · Synthesis
LLM patterning2023 · LLM
Hypothesis search2023 · Synthesis
Program sampling2024 · Synthesis
CodeIt2024 · Synthesis
TTT2024 · Test-time
Product of Experts2024 · Test-time
LPN2024 · Test-time
Evolutionary synthesis2024 · Evolution
BARC ensemble2024 · Synthesis
o3 guided search2024 · Search
CompressARC2025 · Compress
Graph exploration2025 · Search
CombinationMethodological analogy
- DSL search → Hypothesis searchmoves the explicit rule into language + code
- Hypothesis search → Program samplingscales execution-filtered sampling
- Program sampling → Evolutionary synthesisadds iterative selection and repair
- DSL search → BARC ensemblesupplies an explicit-program branch
- Program sampling → BARC ensemblesupplies neural program induction
- LLM patterning → TTTcontrasts frozen prediction with per-task adaptation
- TTT → Product of Expertscombines adaptation with multi-view agreement
- TTT → LPNmoves adaptation into a latent program
- DreamCoder → CodeItshares a learn-from-search loop
- CodeIt → Evolutionary synthesisturns candidate feedback into refinement
- Program sampling → o3 guided searchreplaces broad sampling with a learned guide
- o3 guided search → Graph explorationcompares search over traces with search over world states
The technique map
Methods placed by induction vs. transduction and frozen vs. test-time adaptation.
SynthesisTest-timeEvolutionCompressSearchLLM
The generalization boundary
What each evaluation surface establishes—and what remains outside its evidence.
| Evidence surface | What it proves | What it does not prove |
|---|---|---|
| Observed transitions | The model can replay evidence already seen | It predicts unseen states |
| Unseen states in one game | Within-environment dynamics generalize | The representation transfers |
| Unseen public games | Some cross-game adaptation | Resistance to public-set tuning or contamination |
| Semi-private / private games | Held-out benchmark transfer | Broad agentic intelligence outside ARC |
Agent architecture atlas
Five recurring ways ARC-AGI-3 agents represent, explore, remember, verify, and plan.
Learned action effects
StochasticGoose
- Represent
- CNN transition/value estimates
- Explore
- Prioritize likely progress
- Remember
- Replay buffer
- Verify
- Observed reward and transition
- Plan
- Informed search
Evidence boundary: Model-dependent
VLM tool harness
The Duck / Reki / forge
- Represent
- Images, grids, segments, notes
- Explore
- Model-selected probes
- Remember
- Summaries, queues, eviction
- Verify
- Guards and visual checks
- Plan
- Short-horizon tool use
Evidence boundary: Public/private milestone evidence
Executable world model
EWM
- Represent
- Persistent Python simulator
- Explore
- Resolve model contradictions
- Remember
- Code plus transition log
- Verify
- Exact replay
- Plan
- Simulate before acting
Evidence boundary: Public set only
Ontology-guided model
OPINE-World
- Represent
- Object-centric program
- Explore
- Prioritize ontology error
- Remember
- Program plus counterexamples
- Verify
- CEGIS replay
- Plan
- Model-based
Evidence boundary: Public set only
Research experiment board
Questions converted into interventions, measurements, falsifiers, and required receipts.
| Question | Intervention | Measure | Falsifier | Required receipt |
|---|---|---|---|---|
| Do executable models improve held-out generalization? | Text-only vs executable vs verified executable agents | Private-set RHAE, solves, actions, tokens, wall time | No consistent private-set gain after cost matching | Fresh workspaces, prompts, code, transition logs, per-game seeds |
| Does ontology error choose better experiments? | Random, novelty, disagreement, and ontology-error action selection | Actions to first correct predictive model | No gain across unseen mechanics | Per-action hypotheses, predictions, observations, posterior changes |
| Which memories transfer without leaking game identity? | No memory, episodic traces, compressed mechanics, learned retrieval | Cross-environment efficiency under identity masking | Transfer disappears on private families | Memory snapshots, retrieval logs, contamination audit |
| Where do humans spend their learning actions? | Align first-run human and agent progression by level | Actions to insight, post-insight execution slope, abandonment point | Agent and human curves differ only by a constant scale | Anonymized step replays and a declared event-labeling protocol |
Evaluation receipts
Scores shown with split, protocol, cost, artifacts, and claim boundary.
| Claim | Result | Split | Protocol | Cost | Artifacts | Boundary |
|---|---|---|---|---|---|---|
| Human public corpus | 145 solves / 342 plays | 25 Public Demo | First-run, one attempt | 90-minute sessions | Step replays | Human cohort; not AI performance |
| EWM · GPT-5.5 high | 58.12% mean per-game RHAE | 25 public games | Fresh agent and workspace per game | Not normalized here | Code and run artifacts | Private validation untested |
| OPINE-World | 78.4 action efficiency | 25 public games | No per-game training | Not normalized here | Paper-linked implementation evidence | Public set only |
| Verified EWM · gpt-5.6-sol | ~99% RHAE | 25 public games | Exploratory follow-up | Higher-resource verification | Ablation report | Public-set saturation; model postdates games |
Human learning across public environments
Completion rates for all 25 public environments, preserving the unequal shape of first-run difficulty.
Failure diagnosis map
Observable behavior connected to a likely mechanism and a discriminating repair.
ObservedExplains a local visual effect but cannot predict the next state
DiagnoseNarrative recognition without a causal transition model
Test nextRecord the transition, make a falsifiable prediction, then probe the smallest disagreement
ObservedImports a familiar game rule despite contradictory observations
DiagnoseA prior is treated as fact rather than a hypothesis
Test nextList competing hypotheses and choose an action whose outcomes separate them
ObservedSolves one level, then rediscovers the same mechanic later
DiagnoseUseful knowledge is not consolidated into durable memory
Test nextStore a compact rule, its evidence, and the conditions under which it applies
ObservedEventually completes the environment after many redundant actions
DiagnosePlanning is disconnected from the human-normalized action budget
Test nextTrack information gained per action and stop experiments once the decision changes
Executable-world-model ablation board
Four nested treatments separate persistence, simplification, and exact replay verification.
| Treatment | Persistent model | Simplification | Exact replay | Finding |
|---|---|---|---|---|
| Textual baseline | No | No | No | Can beat a flexible executable model in some settings |
| Flexible executable model | Yes | No | No | Persistence alone is not reliably beneficial |
| + scheduled simplification | Yes | Yes | No | Improves 3 of 4 main model-effort settings |
| + fixed interface and verification | Yes | Yes | Yes | Ranks first in all 4 main settings; uses more resources |
Reproducibility index
A compact inventory of code, prompts, artifacts, environment definition, and held-out evidence.
| System | Code | Prompts | Run artifacts | Environment | Held-out evidence | Grade |
|---|---|---|---|---|---|---|
| EWM | Open | Open | Open | Documented | Not tested | A− |
| OPINE-World | Paper-linked | Described | Partial | Documented | Not tested | B |
| EWM ablation | Paper-linked | Treatments specified | Reported | Documented | Not tested | B+ |
| Milestone #1 agents | Open submissions | Varies | Varies | Competition | Semi-private | B |
| Human corpus | Dataset | Protocol open | 342 replays | 25 public | First-run humans | A |