The papers behind ARC-AGI — and the agentic turn.

By 2019 nearly every AI benchmark rewarded skill at one task — playing Go, classifying images, answering trivia — and Chollet's central move is to insist that skill is not intelligence. Skill is the product of intelligence and prior knowledge plus experience, so a system with enough training data, compute, or hand-coded priors can reach superhuman performance on almost any single task without showing any general capacity to learn. Measuring intelligence by task score therefore lets researchers "buy" skill and systematically confuse the map (a benchmark number) with the territory (the capacity to acquire new skills).

The constructive core is a formal, information-theoretic definition grounded in Algorithmic Information Theory: intelligence is skill-acquisition efficiency over a scope of tasks, controlling for priors, experience, and generalization difficulty. A system needing a million examples is less intelligent than one needing three, even if both eventually reach the same skill. Chollet fixes fair priors as Core Knowledge — the innate systems developmental psychology attributes to humans: objectness and elementary physics, agentness and goals, number and arithmetic, geometry and topology. He operationalizes the measure with the Abstraction and Reasoning Corpus: about 1,000 hand-built visual puzzles, each a few input/output grid pairs (up to 30×30, small palette). From roughly three-to-five demonstrations the solver must infer the transformation rule and apply it to a held-out input, with a private test set for honest scoring.

The paper is a measurement contribution, not a solver: its result is the definition, the benchmark, and a quantified gap — humans solved the large majority of evaluation tasks while the best automated methods solved only a small fraction. The bet proved durable; ARC resisted the pretraining-scaling era far longer than task-specific benchmarks, exactly as the theory predicted. It reframed "AGI" from a vague aspiration into a measurable gap between human and machine generalization, and defined the yardstick that ARC-AGI-2, ARC-AGI-3, and the ARC Prize have extended rather than replaced.

In late 2024 OpenAI's o3, using large amounts of test-time compute, reached the high 70s to high 80s on ARC-AGI-1's semi-private set. The ARC-AGI-2 paper later estimated its retail-equivalent cost at roughly $200 to $20,000 per task. Rather than close the challenge, the authors argued that ARC-AGI-1 had become saturated and offered too little resolution near the top. ARC-AGI-2 is a redesigned, harder benchmark that restored a measurable gap and made efficiency a first-class axis.

The method is human-in-the-loop and adversarial toward frontier models. It preserves the grid-pair format so existing tooling carries over, then selects tasks targeting four demands that defeated o3: multi-rule composition, sequential reasoning, context-dependent control flow, and in-context symbol definition. Candidate tasks go through live human testing; tasks too few humans can solve are discarded, while model-easy tasks provide little signal. A private/semi-private split limits tuning, and the leaderboard reports each system in cost-per-task and score space.

In the launch-era semi-private evaluation, o3 (medium) and o3-mini (high) scored around 3%, the 2024 ARChitects winner around 2.5%, and o1-pro (low) and Claude 3.7 around 0.9%. The offline, fixed-compute ARC Prize 2025 competition drew about 1,455 teams; its best entry reached roughly 24% with synthetic-data generation and test-time training on a 4B-class model, leaving the 85% grand-prize bar unclaimed. ARC-AGI-3 carries the same emphasis on intelligence measured jointly with efficiency.

ARC-AGI-1 and ARC-AGI-2 tested data-efficient modeling: infer a transformation rule from a handful of static input/output pairs. But static benchmarks can be attacked by training on dense samplings of the task space. ARC-AGI-3 keeps the Core Knowledge, no-language, novelty-first spirit while changing the format entirely: the agent is dropped into abstract, turn-based game environments and told nothing — not the goal, not the controls, not the win condition. It must actively discover even what question it is being asked, targeting the "unknown unknowns."

The benchmark decomposes agentic intelligence into four components that must all be exercised: Exploration, Modeling, Goal-Setting, and Planning & Execution. The environment format is deliberately minimal so difficulty lives in reasoning, not perception: each frame is a 64×64 grid of 16 colors, the world is turn-based and never mutates asynchronously, and the action space is tiny (five keys + Undo + click-a-cell), with only state-changing actions counting. All four pillars collapse into one scalar, RHAE (Relative Human Action Efficiency): for each completed level, the human-to-agent action-count ratio is squared, capped at 1.15, weighted by level, capped by the fraction of levels completed, and then averaged across environments. A studio validates each environment with random-play qualification, a graph of hashed reachable states with per-level win probability, and recording-based replay verification.

During launch-era evaluation, harness-free frontier models scored roughly 0.1–0.5%; the best preview agent, "StochasticGoose," reached about 12.58% through informed search rather than explicit modeling. The authors framed this as a deficit in exploration, hypothesis revision, goal inference, and planning under uncertainty, not an API artifact. By grounding the score in first-contact action efficiency, ARC-AGI-3 turns "acquire a correct world model quickly" into a measurable quantity.

ARC-AGI-1 had gone about five years essentially unbeaten, and the 2024 ARC Prize was the first large public push against it. The question the season answered: what actually moves the needle on genuinely novel tasks — scaling a single model, or something else? The report reframes ARC solving around test-time adaptation, arguing that pure "memorize-fetch-apply" inference is not enough.

The report is a survey of competitor methods organized by the transduction-vs-induction distinction. Induction searches for an explicit program — a hand-built DSL or Python emitted by an LLM — keeping only programs that exactly reproduce every training pair, then runs the survivors on the held-out test input; LLM-guided variants propose likely-correct programs and debug failures to shrink the search. Transduction predicts the output grid directly, typically via test-time training: for each task, LoRA fine-tune a pretrained LLM on that task's demonstration pairs plus heavy augmentation (rotations, reflections, color permutations), then vote among augmentation-consistent candidate outputs. The key empirical finding is that these two strategies solve substantially different task sets, so the best systems run both pipelines and reconcile their outputs.

Private-set state of the art rose from ~33% (end of 2023) to 55.5% over the roughly five-month season. The ARChitects won Kaggle at 53.5% using TTT, augmentation, and a product-of-experts selection criterion; MindsAI hit the season-high 55.5% but stayed closed and was prize-ineligible. Greenblatt's GPT-4o synthesis reached ~42% by sampling and debugging thousands of Python programs per task, and Akyürek et al.'s TTT paper hit ~47.5% and won a paper award. Crucially, no static, non-adapting approach exceeded ~11%. But as a brute-force synthetic-data ensemble reached the low 80s, the report candidly judged ARC-AGI-1 near saturation and attackable — directly motivating the cost-aware ARC-AGI-2 and interactive ARC-AGI-3.

Where o3 saturated ARC-AGI-1 in 2024, ARC-AGI-2 reopened a wide human-machine gap, and the 2025 competition was the first run against it. This report documents the result: participation surged (~1,455 teams, ~15,000 entries, paper submissions nearly doubling to ~90) but scores stayed low. The top private-leaderboard entry reached only ~24% against the 85% grand-prize threshold, which went unclaimed — and it got there cheaply, ~$0.20 per task, using synthetic data plus test-time training on a compact few-billion-parameter model. Humans still solve essentially all evaluation tasks, so the gap is stark.

The report's central diagnosis is the emergence of the refinement loop: a per-task, feedback-guided optimization cycle. Rather than sampling candidates once, the strongest entries propose a solution (a program, a chain-of-thought, or a set of weights), verify it against the training pairs, convert the discrepancy into feedback, and iterate until it fits every demonstration. It taxonomizes three flavors: evolutionary program synthesis (populations refined against a verification fitness, e.g. the Berman and Pang entries); application-layer refinement of commercial models (a verifier wraps a frontier model's chain-of-thought — one Gemini-3-Pro-class harness reportedly climbed from ~31% to ~54% at about $30/task); and weight-space/zero-pretraining refinement, exemplified by the Tiny Recursive Model (TRM): ~7M parameters, no pretraining, ~45% on ARC-AGI-1 but only ~8% on ARC-AGI-2.

Two further themes give the report weight. It introduces knowledge-dependent overfitting as a subtler contamination category: performance is bounded by knowledge coverage, and ARC's own conventions may now be latent in base models — the report notes a model applying the correct ARC color mapping without ever being told the task was ARC. And it marks ARC-AGI as an industry standard (Anthropic, Google DeepMind, OpenAI, and xAI all reported numbers in 2025 model cards) before previewing ARC-AGI-3, the first interactive format change since 2019.

For four years, ARC-AGI-1 — puzzles designed to be easy for humans and hard for machines — resisted large language models. Scores crept from ~0% (GPT-3, 2020) to only ~5% (GPT-4o, 2024). Standard models operate by "memorize, fetch, apply": they retrieve a memorized transformation and apply it, which fails on genuinely novel tasks. The open question was whether frontier models could ever adapt to the unseen.

o3 changed the mechanism rather than the scale. At test time it generates many candidate chains-of-thought — natural-language programs for the specific task — and uses a learned evaluator to search and select among them, an AlphaZero-like generate-and-score loop rather than a single forward pass. Chollet calls this deep learning-guided program search, with the chain-of-thought itself as the artifact of knowledge recombination. The "high" versus "low" efficiency settings correspond to how much test-time search compute is spent; the low-efficiency run used ~172× the compute. ARC Prize evaluated under retail-cost accounting that makes per-task price visible.

o3 scored 75.7% (high efficiency, ~$20/task) and 87.5% (172× compute) on the 100-task semi-private set, and ~82.8% / ~91.5% on the 400-task public set — clearing o1's 32% ceiling and the 53% best prior hand-built method. But the caveats made it a pivotal document: cost ran into the thousands per task against ~$5 for a human, o3 still fails some very easy tasks, and early ARC-AGI-2 testing suggested it would drop under ~30% while humans stay above ~95%. Combined with a ~81% Kaggle ensemble, the result signaled ARC-AGI-1 was saturating and motivated the harder ARC-AGI-2 and the interactive ARC-AGI-3 — plus a restated open-source 85%-at-efficiency prize bar that o3 did not clear.

Each ARC task is a handful of input to output grid pairs plus a test input; grids are small (up to ~30×30) with 10 colors, and every task is novel — you cannot memorize, you must infer the transformation from a few demonstrations. In 2020 the open question was how far pure symbolic search could get with no learning at all, and this solution is the archetypal answer.

The solver represents grids as "images" and maintains, per task, a growing pool of candidate pieces. It seeds the pool with the input(s), then up to a bounded depth applies its library of ~142 unary functions to existing pieces, adding each result — a greedy DAG expansion (a DAG rather than a tree because one piece feeds many functions). After building the pool it searches for a stacking or composition of pieces that exactly matches the training outputs across all pairs; any program consistent with every pair is executed on the test input. Depth is the key knob: depth 2 runs the public eval in ~70 seconds but is limited, while depth 3 with test prediction is far stronger and takes ~9 hours on a single 4-core machine. The core is written in C++ for speed with Python orchestration, running fully offline within Kaggle's compute limits.

It scored ~20% on the private leaderboard for first place as a solo competitor, and ~129/419 tasks at depth 2 on the public evaluation set. Historically it is a landmark: it established the canonical ARC-AGI-1 style — hand-built DSL, exhaustive/greedy compositional search, exact-match-on-training-pairs verifier — that dominated until 2024's test-time training and LLM-guided synthesis. Its ceiling, a solo human authoring primitives and a search that explodes combinatorially with depth, is exactly the limitation learned DSLs, neural-guided search, and LLM program generation set out to break.

In-context learning plateaus badly on genuinely out-of-distribution reasoning like ARC: a model can see a few input/output grid pairs but cannot reliably induce the underlying transformation. This paper asks whether test-time training — briefly updating the model's own parameters at inference from a self-supervised loss on the task's own data, then throwing those weights away — can close the gap between what an LM knows and what it can actually do on structurally novel problems.

For each test task the system builds a per-task training set purely from the demonstration pairs using a leave-one-out scheme: hold one pair out as the query while the rest serve as in-context support, then multiply by invertible augmentations (D8 rotations, reflections, transpose, plus color permutations). A fresh LoRA adapter is trained on this synthetic set for a few steps and discarded afterward. It all sits atop a base model first fine-tuned on ARC-like data (ReARC/BARC-style generators). At decode time the input is passed through the same invertible transforms, predictions are un-transformed back to the canonical frame, and a two-stage hierarchical self-consistency vote picks the answer. Ablations isolate three necessary ingredients — initial fine-tuning, the leave-one-out augmented format, and per-instance adapters — each responsible for a measurable chunk of accuracy.

The payoff: an 8B LM reaches 53.0% on ARC public validation with TTT, up to 6x over a fine-tuned baseline, and 61.9% when ensembled with the BARC program synthesizer — matching reported average human performance (~60.2%). The recipe transfers, beating 10-shot prompting on BIG-Bench Hard by 7.3 points (50.5% → 57.8%). The broader message challenges the frozen-weights-plus-long-context orthodoxy: weight adaptation at test time is a distinct axis of test-time compute. A caution the note flags is that these ARC-AGI-1 gains collapse to single digits on ARC-AGI-2, so adaptation must be paired with strict verification.

ARC solving in 2024 faced a hard practical question that headline scores obscured: what wins under the competition's real constraints — fully offline, fixed consumer hardware, bounded time, and roughly two cents per task? The ARChitects' answer is that the decisive lever is not a bigger model or more search, but perspective. The same task, viewed under many equivalent transformations, gives a model many partially-independent readings, and combining those readings correctly is what separates a mediocre single-shot solver from a state-of-the-art one.

The pipeline starts from an open Mistral-NeMo-Minitron-8B base, fine-tuned on augmented ARC-style data, then applies test-time training: for each evaluation task a fresh rank-32 LoRA adapter is fit in about 51 seconds on an RTX 4090, specializing the model to that task's transformation — the single largest accuracy jump, as in the MIT TTT work. Around it sit two engineered pieces. Candidate generation replaces wasteful stochastic sampling with a depth-first token sampler that expands the output grid and prunes any branch whose cumulative probability drops below a threshold T, emitting only about nine candidates per task. Scoring is the named idea: each candidate is transformed into many invertible perspectives, scored under each, and the probabilities multiplied (a product of experts), with a theoretical bound relating the ensemble's divergence to the average single-view divergence plus a gain when views disagree.

The result: 1st place at 53.5% on the private set, and 71.6% on ARC-AGI-1 public eval (about 286/400) with the post-competition pipeline — above the ~60% average-human mark and state of the art among open methods, though below concurrent o3 (~82.8%) at far higher cost. A clean ablation ladder (baseline → TTT → multi-augmentation inference → PoE → DFS) quantifies each step, and ~73% on ConceptARC argues the gains are not pure in-distribution memorization. It is the strongest public evidence that augmentation-as-ensembling, not scale, was 2024's highest-leverage idea.

By mid-2024, ARC-AGI still stumped frontier LLMs, and a common framing held that models like GPT-4o simply could not synthesize genuinely novel solutions. A second, more practical obstacle sat underneath: GPT-4o's grid vision is poor — it reliably misreads grids larger than about 12x12 and struggles even at 8x8 — so any approach has to fight the model's perception, not just its reasoning.

Greenblatt's recipe sidesteps both with brute-force sampling under an execution filter. Each task is packed into a ~30k-token few-shot prompt that renders the training pairs in several redundant encodings at once — 2D arrays with spreadsheet-style cell coordinates, connected-component object listings, normalized shape descriptions, and cell-level diffs — with two prompt families for size-preserving versus size-changing tasks. GPT-4o then emits roughly 8,000 candidate Python transform functions; each is executed against every training pair and discarded unless it reproduces all of them exactly. A revision stage takes the dozen most promising failures, shows the model their concrete wrong outputs beside the expected ones, and draws ~3,000 repair samples. Surviving programs run on the test input, and the top-3 by majority vote are submitted.

The result was ~50% on a 100-task public-eval sample and ~42% on the compute-limited ARC-AGI-Pub set — state-of-the-art at the time, up from the low-30s, with no fine-tuning. Revision alone contributed ~13 points, a far cheaper path than blind resampling (reaching 50% from first-attempt samples would need on the order of 100,000 draws). Accuracy scales at roughly +3% per doubling of samples, so pushing to ~70% would demand millions of draws — the method costs about 1000x more test-time compute than prior work and is ineligible for the official prize. But the lesson stuck: GPT-4o's program distribution places real probability mass near correct solutions, which is exactly why filtered sampling works where random search would not.

Greenblatt had shown that sampling thousands of Python programs and filtering by execution reaches ~43% on ARC-AGI-Pub, but first-shot sampling plateaus: near-correct programs that solve different subsets of examples are discarded. Berman asks whether iterated refinement—treating candidate programs as an evolving population—beats simply drawing more one-shot samples within the leaderboard's ~12-hour window and ~$10,000 API budget.

The base model is never fine-tuned. For each task, Claude 3.5 Sonnet emits an initial generation of ~250 Python transforms. Each runs against the training pairs and receives a two-tier fitness score: exact grids solved, then per-cell correctness. The fittest become parents for revision prompts that breed improved children over up to four generations. A pooled track also combines functions that solve disjoint subsets, preserving diversity against premature convergence.

The system reached a record 53.6% (268/500), roughly 10.6 points above Greenblatt's result and unbeaten until o3. Four generations of 50 reached ~75% versus ~70% for one generation of 200 on 60 training tasks; ~42% of wins emerged in generations 2–4. The result suggests that refinement, not only sample count, contributes. A 2025 follow-up evolves natural-language solution descriptions instead of Python.

The ARC-solving field had split into two camps: inductive program synthesis (search for a rule f that reproduces the demos, then apply it to the test input) versus transduction (predict the held-out output directly with a neural network, no program ever materialized). Each side claimed superiority. This paper asks the cleanest possible version of the question — hold architecture, training data, and base model fixed, and let only the strategy differ — then compares the two head-to-head on ARC.

Both models are fine-tuned from Llama-3.1-8B-Instruct, with grids serialized one token per pixel. To build the training corpus, the authors hand-write ~100–160 "seeds," each a natural-language description, a transform_grid function (the latent f), and a generate_input sampler. An LLM then remixes them in three stages: recombine descriptions self-instruct style, generate code by retrieval over similar seeds, and execute the samplers to materialize fresh input/output grids — expanding 160 seeds into 200k tasks (ARC-Heavy) and up to 400k (ARC-Potpourri). At test time, induction samples up to ~20k candidate programs, keeps those consistent with the training pairs, and applies a survivor; transduction decodes by beam search. The ensemble tries induction — self-verifying against the demos — and falls back to transduction only when no consistent program is found.

The two strategies fail on genuinely different tasks (a stable Venn split of ~26 induction-only, ~35 transduction-only, ~19 solved by both; p < .004), so combining them with test-time training and reranking reaches 56.75% on ARC public validation — surpassing prior published methods and nearing average-human ~60.2% (best human ~97.8%). Induction excels at precise computation and composing concepts; transduction at perceptual judgments, corroborated on ConceptARC. Performance saturates with more human seeds but keeps scaling with synthetic data and compute; a later Akyürek et al. follow-up reused these BARC models to reach 61.9%.

Symbolic program synthesis searches a hand-designed domain-specific language; neural methods lean on heavy stochastic sampling or expensive per-task weight fine-tuning. Both give something up: the DSL is brittle and human-authored, while fine-tuning is costly and never structures a real adaptation mechanism. LPN asks whether test-time search can instead be learned and made part of the architecture, dropping the DSL entirely.

LPN uses two transformers. An encoder maps an input-output pair (x, y) to a distribution over a continuous latent z; a decoder maps a latent and a new input to an output, decoded pixel by pixel. Each z names an implicit program. Training is a leave-one-out variational scheme: for each pair, the encoder sees all the other pairs, and the decoder must reconstruct the held-out output — preventing the encoder from copying the target and forcing the latent to capture a reusable rule. Crucially, in the "Grad k" regime the model is trained with k gradient-refinement steps applied to the latent before decoding, so it learns a landscape actually navigable by gradient ascent. At test time you initialize z from the encoder, take gradient-ascent steps to maximize the log-likelihood of the training pairs, and decode the test input with the refined latent.

Evaluated on programming-by-example tasks and on ARC-AGI (a 178M-parameter model with a 256-dimensional latent, trained on ~51M RE-ARC pairs), LPN matches or beats in-context learning and test-time-training baselines — 78.5% in-distribution vs 57% for TTT — and its out-of-distribution ARC accuracy roughly doubles, from ~7.75% to ~15.5%, once test-time gradient search is enabled. Absolute OOD numbers stay low, but the headline is architectural: building search into the model gives structured, cheap per-task adaptation without a DSL or fine-tuning.

Inductive reasoning means inferring a general transformation rule from a few input/output examples and applying it to a novel input. The paper argues that direct answer prompting works on easy inductive tasks but struggles on structurally novel benchmarks such as ARC. Drawing on Bayesian models of human inductive learning, the authors instead use an explicit hypothesis space: propose candidate rules, then test them.

The method runs Generate → Select → Implement → Validate. GPT-4 proposes ~64 natural-language hypotheses from serialized grids. Each surviving hypothesis becomes a Python program tested against the training pairs, with up to three rounds of execution-feedback repair. LLM summarization or human selection reduces the candidate set. Training verification is useful but not conclusive; the main bottleneck is proposing a correct hypothesis, which GPT-4 did for only 49 of 100 ARC tasks.

On a 100-task ARC subset, accuracy rises from 17% with direct prompting to 23% with programs, 30% with automatic summarization, and 33% with human selection. Gains also appear on 1D-ARC, SyGuS strings, and List Functions. The LLM acts as a proposal distribution over programs, while an external executor supplies feedback.

In ARC programming-by-examples, a code model almost never samples the exact target program by chance, so ordinary reinforcement or rejection sampling receives almost no useful reward. CodeIt's insight is that a failed program still solves some task—just not the requested one.

In each meta-iteration, a 220M-parameter CodeT5+ policy samples up to 24 programs per task. Valid programs are executed and hindsight-relabeled: their realized output grids become the goal, creating replay data. The policy then fine-tunes on replay experiences, mutated programs, and ground-truth seeds. Prioritized sampling emphasizes experiences that match real demonstrations, and sparse object-centric text keeps sequences short.

After 100 meta-iterations, CodeIt solves 59 of 400 evaluation tasks (~15%), reported as the state of the art among full-evaluation methods at the time. Without hindsight relabeling it reaches 24/400; without pretraining, 9/400. The method is not prior-free—it requires a hand-built DSL, interpreter, and seed programs—but it turns sparse program-search feedback into a denser self-improvement signal.

Most ARC methods rely on pretraining, synthetic corpora, program search, or heavy test-time compute. CompressARC tests a narrower claim: can a per-puzzle compression objective produce useful solutions without those ingredients?

For each puzzle, a fresh ~76K-parameter network trains from scratch with the answer grid removed. A VAE-style objective minimizes the bits needed to describe the puzzle: latent KL plus reconstruction cross-entropy. The most compressed model reconstructs the missing cells. An equivariant architecture encodes example-order, color, and rotation/reflection symmetries through weight tying. Each puzzle runs for ~2,000 Adam steps, about 20 minutes on an RTX 4070, and pass@2 reports two guesses.

The method reaches ~34% pass@2 on the ARC-AGI-1 training set and ~20% on evaluation. On ARC-AGI-2 it scores about 4%. Lower-bit solutions correlate with higher accuracy, so compression also provides a confidence signal. Its published limits—counting, long-range structure, topology, and dynamics—make clear where compression alone stops.

ARC-AGI-3 replaces static puzzles with turn-based games that provide no instructions, stated rules, or stated goal. The agent observes a frame, submits an action, receives a new frame, and must infer the mechanics through interaction. This paper shows that a training-free algorithmic explorer can beat model-controlled agents because it can use far more of the roughly 96,000-step interaction budget.

A vision front-end segments single-color connected components, masks the status bar, and ranks targets by size, morphology, and color. A directed graph records each visited frame-state and tried action. The policy first tries an untested local action, then plans a path to the nearest state with an untested action, and finally relaxes its priority threshold. It supports arrow, click, and combined action spaces.

The agent placed third on the ARC-AGI-3 Preview Challenge private leaderboard, solved a median of about 30 of 52 levels in full eight-hour runs, and solved 12 private levels in its official submission. Performance degrades as the state graph grows, and the agent records transitions without inducing a rule that explains them.

Inductive program synthesis usually needs two things handed to it: a domain-specific language of primitives and a strategy for searching it. DreamCoder's wager is that both can be learned. Instead of fixing a DSL up front, the system starts with a few primitives, grows its own library, and trains a neural network to guide search through it.

A wake-sleep loop alternates between solving problems and reorganizing knowledge. In the wake phase, a recognition model predicts which library components may help and guides enumerative search. In the abstraction phase, compression promotes reused sub-computations into named concepts. In the dreaming phase, the recognition network trains on solved tasks and synthetic examples sampled from the current library.

Across domains including list processing, symbolic regression, drawing, and block towers, DreamCoder rediscovers reusable concepts from primitives. Its libraries remain readable, and later concepts build on earlier ones. The pairing of a learned DSL with a learned search policy provides a template for neuro-symbolic ARC solvers and for world models synthesized from interaction.

ARC-style abstract reasoning and low-level sequential control look like different problems. This paper argues they are, from a model's perspective, the same capability: completing structured token sequences. It reframes what a pretrained LLM is — not a language model but a general sequence pattern machine — and asks how much abstract-reasoning competence a frozen, off-the-shelf model already carries with no fine-tuning at all.

Everything runs through in-context prompting on a frozen model. Structured data is serialized to token sequences: ARC grids become rows of number/character tokens laid out as ASCII art, with demonstration input to output pairs concatenated in the prompt and the model asked to continue the held-out output. The authors organize the capability into three modes — sequence transformation (map input tokens to output tokens, how they cast ARC), sequence completion (extrapolate procedurally generated PCFG and functional patterns), and sequence improvement (condition continuations on returns/rewards to extrapolate better trajectories, an in-context form of trial-and-error). For the pivotal token-invariance study, the alphabet is permuted to arbitrary vocabulary tokens so surface semantics are stripped while relational structure survives.

A frozen LLM solves a non-trivial subset of ARC tasks zero-shot with no visual encoder and no ARC-specific training, and its pattern completion partially survives random token remapping — the paper's central evidence that the skill is structural, not memorized. Pushed into robotics, the same machinery extrapolates motions from keyframes and, via least-to-most reward-conditioned prompting, discovers a closed-loop CartPole controller. The authors are candid about practical non-viability today — latency, context limits, compute — and present the work as a capability demonstration, not a deployable system. The through-line for ARC research: a frozen LLM already contains a surprisingly general, semantics-agnostic pattern engine that can be prompted, not trained, to attack abstract reasoning.

The central design choice is to make the world model inspectable and falsifiable. A scripted controller separates observation, coding, replay verification, simplification, planning, and execution. Every newly observed transition can reject the current simulator; the agent must repair a durable program rather than rely only on prose in context.

This improves scientific visibility: a reader can inspect not only the final action but the model that predicted it. The paper also audits unintended information channels and starts every game with a fresh agent and workspace. Its strongest evidence is public-set performance with released artifacts; its strongest limitation is equally clear—private validation and cross-game transfer remain open.

Pixel worlds make program synthesis harder because the ontology itself is unknown. OPINE-World therefore treats representation discovery as part of exploration: one agent interacts with the environment while another proposes and repairs an object-centric simulator. Replay counterexamples drive program revision, and a Bayesian ontology-error measure directs experiments toward places where the current object vocabulary is least adequate.

The result is a tight science loop: hypothesize objects and dynamics, select a discriminating action, compare prediction with observation, repair the program, then plan. The public-set result is strong, but it is still a public-set result; the method's real promise is the explicit coupling between representation error and active exploration.

The study asks an unusually important question for agent research: which component caused the result? Four nested agents isolate persistence, simplification, and verification while holding the task family and controller lineage close. The headline is not that every plausible mechanism helps. A textual baseline beats the flexible executable variant in both GPT-5.5 settings; simplification helps in three of four main settings; the full verification treatment ranks first throughout but costs substantially more.

A gpt-5.6-sol follow-up reaches about 99% RHAE on the public set, yet the author explicitly labels this public-set saturation because the model postdates the games and held-out performance is unknown. The paper is therefore valuable twice: as component evidence and as a model of restrained evaluation language.

In December 2024 OpenAI's o3 reached about 87.5% on ARC-AGI-1, a jump widely reported as a step toward general intelligence. Pfister and Jud accept the number as genuine and impressive, then use the moment to ask what the benchmark actually measures. Their answer turns on a distinction: skills are competence on a bounded, familiar class of problems, while intelligence is the capacity to acquire new competence in unfamiliar situations. On that reading, a very high ARC score demonstrates the former without touching the latter.

The argument hinges on how o3 succeeds. The authors characterize its approach as massive trialling of combinations of predefined operations — a brute-force search over candidate grid transformations, filtered by evaluation at high compute cost. ARC-AGI-1's tasks are solvable by recombining a finite repertoire of operations, so this strategy is highly effective there. But real-world problems, they argue, generally lack a pre-enumerable operation set and lack cheap ways to test candidate solutions in advance, so a recipe built on exhaustive recombination does not extend to open-ended settings. From that gap they advance a positive definition of intelligence: achieving diverse goals in diverse worlds with minimal prior knowledge, centering breadth of transfer and economy of priors over peak performance on any one distribution.

The paper is conceptual, not empirical — no new models or experiments, just a framework and a critique. Under its lens, ARC-AGI-1 is one narrow slice: a single grid representation with a solvable-by-recombination structure, leaving the diversity-of-worlds axis essentially unmeasured even at 87.5%. The authors sketch requirements for a stronger benchmark — far greater task and world diversity, explicit accounting of allowed priors, and a design that forbids solve-by-recombination and pre-testable-solution shortcuts. Read as a corrective to the ARC-Prize narrative, it stays sympathetic to Chollet's skill-acquisition-efficiency framing but insists that clearing one hard benchmark is not the same as measuring general intelligence — a caution pointing toward the interactive, instruction-free setting of ARC-AGI-3.

The paper argues that ARC problems are substantially perceptual and qualitative, and that the transformation space is too broad for hand-guided program search alone. Its alternative is to keep training the network on the task being solved.

Both the network and optimizer become part of inference. A LongT5 encoder-decoder receives row-major numeric grids. Pretraining uses code, multi-task data, synthetic riddles, and an auxiliary DSL-function prediction head. TTFT turns each demonstration pair into held-out training examples, expands them through color, D4, and order augmentations, and then fine-tunes the full network. AIRV applies invertible transforms, predicts in each view, reverses the transforms, and votes.

From a ~5% zero-shot base, AIRV yields roughly +260%; adding TTFT yields a further ~+300%, reaching 58% on the private test set under a single-P100 compute budget. The authors interpret this as evidence for learned, on-the-fly abstraction rather than program search alone.