ARC-AGI terms, defined plainly.

the competition & foundation

The annual public competition and the nonprofit ARC Prize Foundation that stewards the ARC-AGI benchmarks. It runs a Kaggle contest with grand prizes for beating a target score under an efficiency cap, and publishes an annual technical report (2024, 2025).

Abstraction and Reasoning Corpus for Artificial General Intelligence

A benchmark of visual grid puzzles, introduced by François Chollet in 2019, built to be easy for humans and hard for AI. It is designed to make memorization-by-scale less useful by scoring tasks the solver has not seen, emphasizing fluid intelligence over accumulated knowledge. Originally “ARC”; renamed ARC-AGI in 2024.

the original 2019 corpus

The first ARC dataset: 1,000 tasks split into a public training set, a public evaluation set, and a held-back private set. Each task shows a few input→output grid pairs; the solver infers the rule and produces the test output. It resisted the state of the art for roughly five years, until o3.

the 2025 successor

A harder rebuild released in 2025 after o3 nearly solved ARC-AGI-1. It targets compositional, multi-step, and symbolic reasoning, and makes efficiency (cost per task) a first-class metric. Every task was solved by at least two humans, while launch-era pure frontier reasoning models scored in the low single digits.

the interactive / agentic benchmark

The first interactive ARC benchmark (preview July–August 2025; paper 2026). It replaces static grid prediction with a turn-based game that has no instructions, no labeled controls, and no stated goal. An agent must explore, build a world model, infer what winning means, and plan across roughly 135 environments.

the public leaderboard split

The verified public leaderboard configuration for ARC-AGI-1, run on a semi-private evaluation set under a stated compute/cost cap so closed commercial models can post comparable scores. o3’s 87.5% and Berman’s 53.6% are ARC-AGI-Pub numbers.

the assumed innate toolkit

The minimal set of built-in concepts ARC assumes every human shares by about age four: objectness and object persistence, basic geometry and topology, numbers and elementary counting/arithmetic, and agent-like goal-directedness. The public training set exists to teach these, not the answers.

Accumulated, memorized knowledge and practiced skills. Large models trained on the web are rich in it, which is exactly what ARC tries to neutralize so a high score cannot come from having seen the answer before.

MDL · algorithmic information

The compression view of intelligence: the best explanation of the data is the shortest program that reproduces it. This idea underpins the program-synthesis and pure-compression solvers — the shorter the rule that fits every demonstration pair, the more likely it generalizes.

A measure of how novel a task is relative to everything the system’s developers built in — priors plus training data. ARC maximizes it by keeping evaluation tasks unlike anything in the training set or on the public web.

The ability to reason, solve genuinely novel problems, and adapt to new situations without leaning on prior task-specific knowledge. ARC is designed to isolate and measure it — the opposite of crystallized intelligence.

Handling situations that differ from anything in prior experience. ARC specifically demands broad, developer-aware generalization to tasks the system’s own builders never anticipated — not interpolation within a known distribution.

Knowledge a solver is allowed to assume before seeing any task. ARC deliberately bounds priors to a small “Core Knowledge” set so that a score reflects reasoning from those primitives, not domain data baked in during training.

the operational definition of intelligence

Chollet’s core thesis: “The intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty.” Intelligence is not skill itself but how little data and experience a system needs to acquire new skill.

symbol

A single grid square holding one of up to 10 symbols (rendered as colors) in ARC-AGI-1/2, or one of 16 colors in ARC-AGI-3. Colors are categorical labels, not ordered values — blue is not “more” than red.

train pair

An input grid together with its correct output grid, given inside a task as an example — typically two to five per task. The rule must hold for all of them.

The canvas of every task — a 2D matrix of cells, up to 30×30 in ARC-AGI-1 and ARC-AGI-2, and 64×64 in ARC-AGI-3.

A held-back set that is never published, used for the official Kaggle competition leaderboard so that scores cannot be inflated by training on the answers.

Openly released evaluation tasks (400 in ARC-AGI-1) used for self-reported scores, kept separate from the private grading that decides the official leaderboard.

Openly released tasks (400 in ARC-AGI-1) meant to teach the Core Knowledge priors a solver needs — explicitly not a source of the evaluation answers.

A roughly 100-task set introduced in mid-2024 to verify closed, commercial models while limiting how much of the benchmark they are exposed to. It backs the ARC-AGI-Pub leaderboard.

puzzle

One puzzle: a small set of demonstration pairs plus one or more test inputs. The solver infers the single transformation consistent across the demonstrations and produces each test output exactly.

The held-out input grid whose output the solver must generate. Grading is exact-match on the full output grid — a single wrong cell fails the attempt.

the latent program

The hidden procedure that maps input to output and stays consistent across a task’s demonstration pairs. Discovering it — whether as an explicit program or an implicit prediction — is the task.

compute budget · efficiency

The dollars (or compute) a system spends to solve one task. ARC-AGI-Pub and ARC-AGI-2 report it beside accuracy, because a score bought with an unbounded budget proves little — o3’s high score cost thousands of dollars per task.

Running a model on the raw benchmark with no task-specific scaffolding. In launch-era ARC-AGI-3 results, frontier models scored below 1% harness-free — even when a hand-built harness reached ~97% on a seen environment, revealing memorization rather than generalization.

The success rate of a human panel, used as the human reference point—about 98–100% on ARC-AGI-1 and ARC-AGI-2, and 100% on ARC-AGI-3. Every ARC-AGI-2 task is solved by at least two people.

ARC Prize’s verified public ranking of scores, reported under stated compute and cost limits so that accuracy and efficiency are compared on the same footing.

mid-competition award

A paid checkpoint awarded before the final competition closes. ARC-AGI-3’s first 2026 milestone named The Duck, Reki, and forge, but it did not decide the final 2026 podium.

the harness boundary

ARC-AGI-3’s official leaderboard measures general models without an environment-specific harness; the community surface can study custom scaffolding. The distinction is essential: the technical report shows a seen environment moving from 0% harness-free to 97.1% with a hand-built harness without transferring to another hidden game.

the official metric

ARC’s grading rule: a task counts as solved if either of two allowed attempts exactly matches the answer grid. Reported scores are the fraction of tasks solved at pass@2.

Relative Human Action Efficiency

ARC-AGI-3’s score. For each completed level it squares the human-to-agent action-count ratio to punish wasted actions, caps that level term at 1.15, then applies level weighting and benchmark aggregation — a first-contact comparison of human vs. AI learning efficiency, not just success.

CompressARC

Solving a task with no pretraining by compressing the grids themselves: the model that best compresses a task has implicitly captured its rule. A direct, striking instantiation of the description-length view.

wake-sleep program learning

A program-synthesis system that grows a reusable library of abstractions by alternating wake phases (solve tasks) with sleep phases (invent new abstractions and “dream” practice problems). A foundation for the synthesis line.

domain-specific language

A hand-built vocabulary of grid operations (recolor, mirror, crop, flood-fill…) that a search composes into a solving program. The backbone of Icecuber’s 2020 Kaggle-winning solution.

test-time compute

Iteratively mutating and selecting candidate solutions against the demonstration pairs, using test-time compute to improve them. Berman’s method reached a record 53.6% on ARC-AGI-Pub.

CodeIt

A self-improvement loop that relabels a program’s actual (wrong) output as if it had been the goal, then replays these examples with prioritization — letting the model learn from failures instead of discarding them.

Having an LLM first generate several natural-language hypotheses about the rule, then translate only the promising ones into executable programs — separating “what might the rule be” from “write the code.”

Having an LLM infer the rule purely from examples placed in its prompt, with frozen weights and no fine-tuning. The lightest-weight way to attempt a task.

the inductive route

Inferring an explicit, reusable rule or program from the examples and then applying it. Contrasts with transduction, which skips the explicit rule.

LPN

Learning a continuous latent space of programs and searching it by gradient descent at test time, rather than enumerating discrete code.

Prompting a language model to write many candidate programs (often thousands of Python snippets), then keeping those that reproduce the demonstration pairs. Greenblatt’s route to 50% with GPT-4o.

Combining many transformed “views” of a task multiplicatively so that only answers consistent across all perspectives survive — a way to boost an LLM’s reliability on ARC.

Solving a task by finding a program — in a DSL or in general code — that reproduces every demonstration pair, then executing it on the test input. The central paradigm of the ARC solver lineage.

TTT · test-time adaptation

Fine-tuning the model on a task’s own demonstration pairs (and augmentations of them) at inference, just before it answers — turning each puzzle into a tiny training run. Surprisingly effective on ARC.

the transductive route

Predicting the test output directly from the demonstration pairs, without ever writing down an explicit intermediate program. The two routes are complementary — the strongest systems combine them.

The small vocabulary from which each environment exposes a subset: five keyboard actions, Undo, and click-a-cell. The world is turn-based and never moves on its own, so the challenge is reasoning rather than reflexes.

One ARC-AGI-3 game or world. The technical report defines 135 in the benchmark: 25 Public Demo, 55 Semi-Private, and 55 Fully Private. Each hides its own mechanics, controls, and win condition; later levels compose ideas discovered earlier.

capability 1

Gathering information through action when there are no instructions to read. The first ARC-AGI-3 capability: probe the environment to discover its rules.

The 64×64 grid of 16 colors the agent observes on each turn of an ARC-AGI-3 environment — the interactive analogue of a static ARC grid.

capability 3

Figuring out what “winning” even is when the objective is never stated and no reward arrives until the win frame. The third ARC-AGI-3 capability.

Scaffolding wrapped around a model — memory, tools, prompt structure, or hard-coded strategy. Harnesses are valuable engineering artifacts, but ARC-AGI-3 separates them from raw-model evidence: 97.1% on one seen environment can coexist with 0% harness-free and no transfer.

A benchmark where the solver acts and observes over many turns instead of reading one fixed problem. ARC-AGI-3’s defining shift away from the static input→output format used since 2019.

capability 4

Choosing actions toward the inferred goal and revising the plan when observations contradict the model. The fourth ARC-AGI-3 capability depends on exploration, modeling, and goal inference.

A compact, periodically rewritten account of what an agent believes about the game, used instead of retaining every full frame and reasoning trace. Reki and forge carried reflection memory across play in the first 2026 ARC-AGI-3 milestone.

Tufa Labs

The 2025 ARC-AGI-3 Preview Challenge winner at 12.58%. A CNN+RL model predicts which actions change the frame and uses that signal to guide search. It is a preview result, not the 2026 final competition.

Tufa Labs · ARC-AGI-3 Milestone #1

The first-place 2026 ARC-AGI-3 Milestone #1 agent. A local Qwen 3.6 27B FP8 model inspects rendered images, raw grids, and segmented regions, writes and runs Python in a live REPL, and evicts old context to keep playing. It is a milestone winner; the final season remains open.

An internal, predictive model of an environment’s dynamics—“if I press X, the frame becomes Y.” Building one from interaction is the second of ARC-AGI-3’s four required capabilities, and a capability current agents still struggle to learn.

CoT

Generating intermediate reasoning steps before the final answer. It is the mechanism behind reasoning models’ ARC-AGI-1 gains — and the focus of the critique that a high ARC-AGI-1 score is not, by itself, evidence of AGI.

A large chain-of-thought model (the o-series and peers) that spends test-time compute “thinking” before answering. Strong on ARC-AGI-1, but weak on ARC-AGI-2 and near-zero on ARC-AGI-3 harness-free.

OpenAI

The model Ryan Greenblatt drove to ~50% on the public evaluation (42% on ARC-AGI-Pub) in mid-2024 by sampling thousands of candidate Python programs per task — the high-water mark before o3.

OpenAI

OpenAI’s late-2024 frontier reasoning model. It scored 75.7% (low-compute) and 87.5% (high-compute) on ARC-AGI-Pub — the first system to reach the human-competitive bar on ARC-AGI-1, and the reason the benchmark was rebuilt.