How the machines solve it — and how far they got.

Why it worksGrids are tiny (≤30×30, 10 colors), so a good hand-crafted library plus depth-limited search covers enough of the transformation space — and exact match on every pair is a clean, high-precision correctness gate.

Where it stopsPrimitives are authored by one human, so coverage caps near 20%; search explodes with depth (~70s at depth 2 → ~9h at depth 3). No learning, no cross-task transfer.

Why it worksThe number of sampled programs is a near-monotone axis (~+3% per doubling): GPT-4o's program distribution places real mass near correct solutions, so execution filtering surfaces working programs where random search never would.

Where it stopsUses ~1000× more test-time compute than prior work (prize-ineligible), steep diminishing returns (~70% would need millions of draws), and false positives that fit train but fail test.

Why it worksEven with matched data and base model, induction wins precise/compositional tasks while transduction wins fuzzy/perceptual ones — they fail on different tasks, so the union approaches average-human performance.

Where it stopsThe complementarity is empirical with no mechanism; induction has a ~9% false-positive rate that caps precision, and transduction can't verify itself. Best ensemble still trails the best humans (~98%).

Why it worksSplitting reasoning into two levels — "what is the rule" (language, good at the gist) then "how to code it" (verifiable Python) — beats jumping straight to code; ablations show both levels help independently.

Where it stopsPerception is the ceiling: GPT-4 barely parses a 2-D grid from text. Hypothesis generation is the bottleneck — a correct rule was produced for only 49/100 tasks — and search is uninformed.

Why it worksARC reward is almost always zero — a model never samples the exact target by chance — so ordinary RL has nothing to learn from. Relabeling manufactures a dense signal; ablations show it is the single most important component.

Where it stopsPlateaus at ~15%. It is not prior-free: it needs a hand-built DSL, an interpreter, and seed programs; reward is exact-match only, and most relabeled goals do not match the original task.

Why it worksFramed as Bayesian program learning: the library is a generative prior, solving is posterior inference, and the neural model is an amortized approximation that makes search tractable — so accuracy and search efficiency compound.

Where it stopsSearch cost grows with library richness; abstraction can't invent a concept never once used, risking local optima; fantasies may drift from the real task distribution.

Why it worksIn-context learning plateaus on out-of-distribution tasks; letting the model briefly rewrite a small slice of its weights explicitly adapts it to the exact task in front of it. A per-instance adapter is a distinct axis of test-time compute.

Where it stopsA fresh adapter per task strains an offline budget; the augmentation trick assumes geometric/color symmetry and degrades on counting or relational rules; ARC-AGI-1 gains collapse to single digits on ARC-AGI-2.

Why it worksEvery perspective is a semantics-preserving view of the same task, so multiplying probabilities enforces cross-view agreement and demotes answers that are only plausible from one angle — many partial readings combine into a strong solver.

Where it stopsNeeds a rich family of invertible perspectives; DFS assumes the answer sits in the high-probability region; ARC-AGI-1 dominance did not transfer to ARC-AGI-2 (low single digits).

Why it worksBecause the decoder is differentiable, test-time search over the latent is cheap gradient ascent instead of stochastic sampling — and training with that same search shapes a landscape that is actually searchable. Adaptation is built into the architecture.

Where it stopsAbsolute out-of-distribution performance stays low (~15%); gradient ascent finds only local optima; the latent program is not inspectable or verifiable like explicit code.

Why it worksThe model emits executable programs, so each candidate is auto-scored against the demos. Depth beats breadth — ~42% of wins came from generations 2–4 — showing iterated refinement, not just more samples, does real work.

Where it stopsFixed lineages waste compute, and per-cell fitness is a heuristic that admits false positives. The deeper "verifier problem": no below-AGI checker reliably confirms correctness, capping filtered evolution.

Why it worksThe Minimum Description Length principle operationalizes intelligence-as-compression: by Occam's razor the shortest description consistent with the visible grids tends to fill the unknown cells correctly — and lower-bit solutions empirically correlate with more accurate ones.

Where it stopsStructurally local: can't count, do arithmetic, extend long-range patterns, or reason about topology. ~4% on ARC-AGI-2, ~20 min/puzzle, and often contains but fails to rank the right answer.

Why it worksInstead of the standard "memorize, fetch, apply" loop, the chain-of-thought becomes the artifact of knowledge recombination — a program the model builds and runs for that task — so guided search gives genuine test-time adaptability, not just a bigger forward pass.

Where it stopsExtreme cost (~$20 to thousands per task vs ~$5 for a human); still fails some very easy tasks; projected under ~30% on ARC-AGI-2; proprietary and short of the open, cost-capped prize bar.

Why it worksA cheap deterministic explorer keeps explicit state-tracking, so it can spend the environment's full ~96,000-step budget productively and never re-tread known paths — where LLM agents stall at ~4,000 model-gated steps.

Where it stopsExploration cost grows with the reachable state space (intractable on large games); exact frame-hashing fragments under cosmetic change; it explores but never induces a rule — it records transitions, it doesn't model them.

Why it worksPerformance remains above chance after random token remapping, suggesting that structural pattern continuation contributes alongside learned token semantics.

Where it stopsToken-invariance is only partial; the solvable subset skews to local/geometric tasks; flattening a grid to 1-D discards 2-D adjacency; everything is bounded by context length and prompt format.