A new research paper investigates the specific architectural components that enable coding agents to tackle the ARC-AGI-3 benchmark. The study seeks to attribute the performance of previous agents to three main ideas: executable world modeling, scheduled simplification, and exact replay verification.
Four Nested Agent Architectures
To isolate the impact of each component, researchers developed four nested Codex-based agents:
- A textual baseline agent.
- A flexible-interface executable world model without replay verification.
- The same executable model with added scheduled simplification.
- A fixed-interface verification treatment requiring exact reproduction of recorded observations.
The study evaluated these variants using gpt-5.4 and gpt-5.5 at high and extra-high reasoning effort levels, with follow-up tests conducted on gpt-5.6-sol at extra-high and maximum effort.
Key Research Findings
The most robust conclusion is that agent performance scales consistently with stronger models and increased reasoning effort. Interestingly, requiring a persistent executable deliverable was not universally beneficial; the textual baseline actually outperformed the flexible-interface executable variant in both gpt-5.5 settings.
While simplification improved performance in three out of four settings, the complete verification treatment ranked first across all four settings, albeit at a higher resource cost. In the gpt-5.6-sol follow-up, the verification variant fully solved every public ARC-AGI-3 game, achieving approximately 99% RHAE and utilizing fewer than half the total actions of the human baseline.
Limitations and Saturation
The researchers caution that these results represent a saturation of the public ARC-AGI-3 games. Since the model postdates the release of these specific games, and performance on held-out data remains untested, the results should be interpreted as mastering a known set rather than a definitive proof of generalized AGI capabilities.
