Every ARC-AGI term, defined.

the competition & foundation

The annual public competition and the nonprofit ARC Prize Foundation that stewards the ARC-AGI benchmarks. It runs a Kaggle contest with grand prizes for beating a target score under an efficiency cap, and publishes an annual technical report (2024, 2025).

Abstraction and Reasoning Corpus for Artificial General Intelligence

A benchmark of visual grid puzzles, introduced by François Chollet in 2019, built to be easy for humans and hard for AI. It resists memorization-by-scale by scoring performance on tasks the solver has never seen — a direct probe of fluid intelligence rather than accumulated knowledge. Originally just “ARC”; renamed ARC-AGI in 2024.

the original 2019 corpus

The first ARC dataset: 1,000 tasks split into a public training set, a public evaluation set, and a held-back private set. Each task shows a few input→output grid pairs; the solver infers the rule and produces the test output. It resisted the state of the art for roughly five years, until o3.

the 2025 successor

A harder rebuild released in 2025 after o3 nearly solved ARC-AGI-1. It targets compositional, multi-step, and symbolic reasoning, and makes efficiency (cost per task) a first-class metric. Every task is solved by at least two humans, while pure frontier reasoning models score in the low single digits.

the interactive / agentic benchmark

The first interactive ARC benchmark (preview July–August 2025; paper 2026). It drops the static grid for a turn-based game with no instructions, no controls, and no stated goal. An agent must explore, build a world model, infer what winning is, and plan across roughly 135 environments.

the public leaderboard split

The verified public leaderboard configuration for ARC-AGI-1, run on a semi-private evaluation set under a stated compute/cost cap so closed commercial models can post comparable scores. o3’s 87.5% and Berman’s 53.6% are ARC-AGI-Pub numbers.

the assumed innate toolkit

The minimal set of built-in concepts ARC assumes every human shares by about age four: objectness and object persistence, basic geometry and topology, numbers and elementary counting/arithmetic, and agent-like goal-directedness. The public training set exists to teach these, not the answers.

Accumulated, memorized knowledge and practiced skills. Large models trained on the web are rich in it, which is exactly what ARC tries to neutralize so a high score cannot come from having seen the answer before.

MDL · algorithmic information

The compression view of intelligence: the best explanation of the data is the shortest program that reproduces it. This idea underpins the program-synthesis and pure-compression solvers — the shorter the rule that fits every demonstration pair, the more likely it generalizes.

A measure of how novel a task is relative to everything the system’s developers built in — priors plus training data. ARC maximizes it by keeping evaluation tasks unlike anything in the training set or on the public web.

The ability to reason, solve genuinely novel problems, and adapt to new situations without leaning on prior task-specific knowledge. ARC is designed to isolate and measure it — the opposite of crystallized intelligence.

Handling situations that differ from anything in prior experience. ARC specifically demands broad, developer-aware generalization to tasks the system’s own builders never anticipated — not interpolation within a known distribution.

Knowledge a solver is allowed to assume before seeing any task. ARC deliberately bounds priors to a small “Core Knowledge” set so that a score reflects reasoning from those primitives, not domain data baked in during training.

the operational definition of intelligence

Chollet’s core thesis: “The intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty.” Intelligence is not skill itself but how little data and experience a system needs to acquire new skill.

symbol

A single grid square holding one of up to 10 symbols (rendered as colors) in ARC-AGI-1/2, or one of 16 colors in ARC-AGI-3. Colors are categorical labels, not ordered values — blue is not “more” than red.

train pair

An input grid together with its correct output grid, given inside a task as an example — typically two to five per task. The rule must hold for all of them.

The canvas of every task — a 2D matrix of cells, up to 30×30 in ARC-AGI-1 and ARC-AGI-2, and 64×64 in ARC-AGI-3.

A held-back set that is never published, used for the official Kaggle competition leaderboard so that scores cannot be inflated by training on the answers.

Openly released evaluation tasks (400 in ARC-AGI-1) used for self-reported scores, kept separate from the private grading that decides the official leaderboard.

Openly released tasks (400 in ARC-AGI-1) meant to teach the Core Knowledge priors a solver needs — explicitly not a source of the evaluation answers.

A roughly 100-task set introduced in mid-2024 to verify closed, commercial models while limiting how much of the benchmark they are exposed to. It backs the ARC-AGI-Pub leaderboard.

puzzle

One puzzle: a small set of demonstration pairs plus one or more test inputs. The solver infers the single transformation consistent across the demonstrations and produces each test output exactly.

The held-out input grid whose output the solver must generate. Grading is exact-match on the full output grid — a single wrong cell fails the attempt.

the latent program

The hidden procedure that maps input to output and stays consistent across a task’s demonstration pairs. Discovering it — whether as an explicit program or an implicit prediction — is the task.

compute budget · efficiency

The dollars (or compute) a system spends to solve one task. ARC-AGI-Pub and ARC-AGI-2 report it beside accuracy, because a score bought with an unbounded budget proves little — o3’s high score cost thousands of dollars per task.

Running a model on the raw benchmark with no task-specific scaffolding. On ARC-AGI-3, frontier models score below 1% harness-free — even when a hand-built harness reaches ~97% on a seen environment, revealing memorization rather than generalization.

The success rate of a human panel, used as the ceiling a system is measured against — about 98–100% on ARC-AGI-1 and ARC-AGI-2, and 100% on ARC-AGI-3. Every ARC-AGI-2 task is solved by at least two people.

ARC Prize’s verified public ranking of scores, reported under stated compute and cost limits so that accuracy and efficiency are compared on the same footing.

the official metric

ARC’s grading rule: a task counts as solved if either of two allowed attempts exactly matches the answer grid. Reported scores are the fraction of tasks solved at pass@2.

Relative Human Action Efficiency

ARC-AGI-3’s score. For each level it is the human action count divided by the agent’s, capped at 1.15×, then squared to punish wasted actions, level-weighted, and averaged over the environments — a first-contact comparison of human vs. AI learning efficiency, not just success.

CompressARC

Solving a task with no pretraining by compressing the grids themselves: the model that best compresses a task has implicitly captured its rule. A direct, striking instantiation of the description-length view.

wake-sleep program learning

A program-synthesis system that grows a reusable library of abstractions by alternating wake phases (solve tasks) with sleep phases (invent new abstractions and “dream” practice problems). A foundation for the synthesis line.

domain-specific language

A hand-built vocabulary of grid operations (recolor, mirror, crop, flood-fill…) that a search composes into a solving program. The backbone of Icecuber’s 2020 Kaggle-winning solution.

test-time compute

Iteratively mutating and selecting candidate solutions, scored against the demonstration pairs, spending more compute at test time to climb. Berman’s record 53.6% on ARC-AGI-Pub.

CodeIt

A self-improvement loop that relabels a program’s actual (wrong) output as if it had been the goal, then replays these examples with prioritization — letting the model learn from failures instead of discarding them.

Having an LLM first generate several natural-language hypotheses about the rule, then translate only the promising ones into executable programs — separating “what might the rule be” from “write the code.”

Having an LLM infer the rule purely from examples placed in its prompt, with frozen weights and no fine-tuning. The lightest-weight way to attempt a task.

the inductive route

Inferring an explicit, reusable rule or program from the examples and then applying it. Contrasts with transduction, which skips the explicit rule.

LPN

Learning a continuous latent space of programs and searching it by gradient descent at test time, rather than enumerating discrete code.

Prompting a language model to write many candidate programs (often thousands of Python snippets), then keeping those that reproduce the demonstration pairs. Greenblatt’s route to 50% with GPT-4o.

Combining many transformed “views” of a task multiplicatively so that only answers consistent across all perspectives survive — a way to boost an LLM’s reliability on ARC.

Solving a task by finding a program — in a DSL or in general code — that reproduces every demonstration pair, then executing it on the test input. The central paradigm of the ARC solver lineage.

TTT · test-time adaptation

Fine-tuning the model on a task’s own demonstration pairs (and augmentations of them) at inference, just before it answers — turning each puzzle into a tiny training run. Surprisingly effective on ARC.

the transductive route

Predicting the test output directly from the demonstration pairs, without ever writing down an explicit intermediate program. The two routes are complementary — the strongest systems combine them.

The tiny, fixed set of inputs available every turn — five keys, an Undo, and click-a-cell — identical across all environments. The world is turn-based and never moves on its own, so the challenge is reasoning, not reflexes.

One ARC-AGI-3 game or world — roughly 135 in the set. Each hides its own mechanics that the agent must discover from scratch; difficulty grows by composing mechanics you had to learn yourself.

capability 1

Earning information by acting when there are no instructions to read. The first ARC-AGI-3 capability — you cannot be told the rules, so you must probe for them.

The 64×64 grid of 16 colors the agent observes on each turn of an ARC-AGI-3 environment — the interactive analogue of a static ARC grid.

capability 3

Figuring out what “winning” even is when the objective is never stated and no reward arrives until the win frame. The third ARC-AGI-3 capability.

Task-specific scaffolding wrapped around a model — memory, tools, prompt structure, hard-coded strategy. ARC-AGI-3 penalizes harnesses that memorize a seen environment but fail to transfer: ~97% seen can still mean 0% harness-free.

A benchmark where the solver acts and observes over many turns instead of reading one fixed problem. ARC-AGI-3’s defining shift away from the static input→output format used since 2019.

capability 4

Routing to the inferred goal and re-planning when reality disagrees with the model. The fourth ARC-AGI-3 capability — and it only works if the first three already did.

Tufa Labs

The winning agent of the ARC-AGI-3 Preview Challenge at 12.58% — a search-and-exploration system. It embodies the frontier’s pattern: the leaders all search the state space; none yet truly models it.

An internal, predictive model of an environment’s dynamics — “if I press X, the frame becomes Y.” Building one from interaction is the second of ARC-AGI-3’s four required capabilities, and the one today’s agents most conspicuously lack.

CoT

Generating intermediate reasoning steps before the final answer. It is the mechanism behind reasoning models’ ARC-AGI-1 gains — and the focus of the critique that a high ARC-AGI-1 score is not, by itself, evidence of AGI.

A large chain-of-thought model (the o-series and peers) that spends test-time compute “thinking” before answering. Strong on ARC-AGI-1, but weak on ARC-AGI-2 and near-zero on ARC-AGI-3 harness-free.

OpenAI

The model Ryan Greenblatt drove to ~50% on the public evaluation (42% on ARC-AGI-Pub) in mid-2024 by sampling thousands of candidate Python programs per task — the high-water mark before o3.

OpenAI

OpenAI’s late-2024 frontier reasoning model. It scored 75.7% (low-compute) and 87.5% (high-compute) on ARC-AGI-Pub — the first system to reach the human-competitive bar on ARC-AGI-1, and the reason the benchmark was rebuilt.