Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?

• URL: http://arxiv.org/abs/2607.15439v1
• Date: Thu, 16 Jul 2026 20:18:59 GMT
• Title: Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?
• Abstract summary: We evaluate the performance of four nested Codex-based ARC-AGI-3 agents.<n>The most robust result is that every agent variant improves with a stronger model.<n>Because the model postdates these games and held-out performance remains untested, this result should be interpreted as saturation of the public set only.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Our previous ARC-AGI-3 agent bundled executable world modeling, scheduled simplification, and exact replay verification, leaving unclear which idea accounted for its performance. We address this attribution question with four nested Codex-based agents: a textual baseline; a flexible-interface executable world model without replay verification; the same executable model with scheduled simplification; and a fixed-interface verification treatment that retains simplification and requires exact reproduction of recorded observations. The main study evaluates all four agents with gpt-5.4 and gpt-5.5 at high and xhigh reasoning effort on the public ARC-AGI-3 games. Exploratory follow-ups evaluate the textual and verification variants with gpt-5.6-sol at xhigh and max. The most robust result is that every agent variant improves with a stronger model and with greater reasoning effort. Within each model-effort setting, differences among variants are smaller than anticipated, while the effects of individual components vary across settings. Requiring a persistent executable deliverable is not universally beneficial: the textual variant outperforms the flexible-interface executable variant in both gpt-5.5 settings. Simplification improves performance in three of the four model-effort settings, with the weakest setting as the only exception. The complete verification treatment ranks first in all four settings, although it uses substantially more resources. In the gpt-5.6-sol follow-up, the verification variant fully solves every public game at both reasoning efforts, achieves about 99% RHAE, and uses fewer than half the total actions of the human baseline. Because the model postdates these games and held-out performance remains untested, this result should be interpreted as saturation of the public set only.

Related papers

• PoTRE: Test-Time Reasoning inspired by Cognitive Heterogeneity [0.24554686192257422]
  PoTRE (Poly-Topological Reasoning Ensembles) is a heterogeneous framework that decouples inference into four agents.<n>We evaluate PoTRE on three frontier benchmarks: ARC-AGI-2, Humanity's Last Exam (HLE), and PRBench Finance.<n>PoTRE achieves state-of-the-art accuracy of 49.92% on HLE, surpassing the previous best official score.
  arXiv  Detail & Related papers  (2026-07-22T15:20:53Z)
• ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations [84.45092853572407]
  ARM is a representation-based AutoRegressive Model that unifies image understanding, generation, and editing.<n>Our tokenizer is supervised with multiple objectives that jointly promote semantic discriminability, language alignment, and faithful reconstruction.<n>Finally, ARM applies reinforcement learning to optimize task-level objectives such as visual quality, instruction adherence, and edit consistency.
  arXiv  Detail & Related papers  (2026-06-09T17:59:28Z)
• FineVerify: Scaling Test-Time Compute with Fine-Grained Self-Verification for Agentic Search [88.16262636915975]
  FineVerify is a fine-grained self-verification framework for agentic search.<n>It decomposes each question into checkable sub-questions, verifies sampled candidates, and selects the candidate with the highest aggregated score.<n>FineVerify consistently outperforms standard scaling baselines.
  arXiv  Detail & Related papers  (2026-05-30T10:21:20Z)
• RewardHarness: Self-Evolving Agentic Post-Training [51.16917192137169]
  RewardHarness is a self-evolving agentic reward framework for instruction-guided image editing.<n>It aligns with human preferences by iteratively evolving a library of tools and skills from as few as 100 preference demonstrations.<n>It achieves 47.4% average accuracy on image-editing evaluation benchmarks, surpassing GPT-5 by 5.3 points.
  arXiv  Detail & Related papers  (2026-05-09T05:32:48Z)
• Executable World Models for ARC-AGI-3 in the Era of Coding Agents [0.0]
  We evaluate an initial coding-agent system for ARC-AGI-3.<n>The system uses a scripted controller, predefined world-model interfaces, verifier programs, and a plan executor.<n>Results provide preliminary evidence that verifier-driven executable world models are a promising approach for ARC-AGI-3 agents.
  arXiv  Detail & Related papers  (2026-05-06T17:12:36Z)
• Three Roles, One Model: Role Orchestration at Inference Time to Close the Performance Gap Between Small and Large Agents [0.4666493857924357]
  We study whether inference-time scaffolding alone, without any additional training compute, can improve the performance of a small model in complex multi-step environments.<n>We evaluate Qwen3-8B on the AppWorld benchmark under both full-precision and 4-bit quantized configurations.<n>On full-precision inference, our scaffolded 8B model surpasses DeepSeek-Coder 33B Instruct (7.1%) from the original AppWorld evaluation.
  arXiv  Detail & Related papers  (2026-04-13T13:40:33Z)
• ClawArena: Benchmarking AI Agents in Evolving Information Environments [61.664633997138004]
  ClawArena is a benchmark for evaluating AI agents in evolving information environments.<n>Each scenario maintains a complete hidden ground truth while exposing the agent only to noisy, partial, and sometimes contradictory traces.<n> evaluation is organized around three coupled challenges: multi-source conflict reasoning, dynamic belief revision, and implicit personalization.
  arXiv  Detail & Related papers  (2026-04-05T17:55:23Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• An Analysis of Architectural Impact on LLM-based Abstract Visual Reasoning: A Systematic Benchmark on RAVEN-FAIR [0.0]
  GPT-4.1-Mini consistently achieved the highest overall accuracy across all architectures.<n>Each model exhibited distinct sensitivity patterns to architectural design, underscoring that reasoning effectiveness remains model-specific.
  arXiv  Detail & Related papers  (2025-11-14T22:50:22Z)
• Model Editing with Canonical Examples [75.33218320106585]
  We introduce model editing with canonical examples. A canonical example is a simple instance of good behavior, e.g., The capital of Mauritius is Port Louis. We propose sense finetuning, which selects and finetunes a few sense vectors for each canonical example.
  arXiv  Detail & Related papers  (2024-02-09T03:08:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.