Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3

• URL: http://arxiv.org/abs/2605.25931v1
• Date: Mon, 25 May 2026 15:13:18 GMT
• Title: Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3
• Abstract summary: We investigate all 25 public ARC-AGI-3 games and find that every one is reachable through non-intelligent strategies.<n>This benchmark critique implies the public evaluation set cannot discriminate intelligent exploration.<n>We present AERA, a three-phase (EXPLORE / VERIFY / PLAN) agent achieving RHAE=0.2116 on these 25 games.
• Score: 0.0
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We systematically investigate all 25 public ARC-AGI-3 games and find that every one is reachable through non-intelligent strategies: 10 in a single blind step, 5 after one probing action, 1 via repeated ACTION1 presses, 1 via diverse exploration, and 8 via single repeated actions with sufficient budget (50-200 steps). A library-level null-coordinate vulnerability additionally bypasses 18 games in 1 step. This benchmark critique implies the public evaluation set cannot discriminate intelligent exploration from trivial heuristics - the private 55-game evaluation is the only genuine intelligence test. Against this backdrop, we present AERA (Adaptive Epistemic Reasoning Agent), a three-phase (EXPLORE / VERIFY / PLAN) agent achieving RHAE=0.2116 (4/25 solved) on these 25 games with Qwen2.5-0.5B, while random and no-explore baselines score 0.0000. We formalise AERA through a Speed--Depth trade-off framework: under a convexity assumption (proved for a class of environments in the Appendix), RHAE's quadratic form emerges as a second-order penalty for deviating from the Pareto frontier between action efficiency and information gain. Contributions: (i) a benchmark validity analysis showing that current interactive reasoning benchmarks fail to measure the exploration they claim to require, and (ii) the EXPLORE-before-PLAN framework and model-capability x exploration interaction. The linked code track entry achieves RHAE=0.30 on the full 55-game private evaluation. Code: CC0.

Related papers

• TRACE: Training Reasoning Agents for Causal Exploration with Synthesized Rewards [15.908097225313968]
  Reinforcement learning with verifiable rewards (RLVR) has advanced language-model reasoning in domains such as mathematics and code.<n>We ask whether this asymmetry of verification can be engineered.<n>We instantiate this approach in TRACE, a digital-advertising diagnostic environment with 12 root causes and fine-grained segment attribution.
  arXiv  Detail & Related papers  (2026-09-09T15:21:47Z)
• Apodex Discovery: Reality Benchmarks and Environments for Evaluating and Building Discoverative Artificial Intelligence [86.61561123749011]
  We introduce Apodex Discovery, a framework for building and evaluating discoverative AI through the heavy-duty solver.<n>First, a problem-scouting process surveyed 561 industries across 16 sectors, assembled 423 high-value real-world problems, and selected 20 for the initial release.<n>Second, a common environment-task-episode abstraction provides data, tools, constraints, feedback, trajectory recording, and verification of intermediate artifacts and final submissions.
  arXiv  Detail & Related papers  (2026-08-11T18:48:40Z)
• The Saturation Trap and the Subjectivity of Intervention Timing: Why Affect-Based Triggers and LLM Judges Fail to Time Interventions on Autonomous Agents [0.0]
  Intervention timing is a low-reliability construct, making single-annotator F1 an unsuitable optimization target.<n>Our contribution is the joint mapping of this problem across human inter-rater reliability, four detector architectures, a cross-model LLM-judge sweep, and a reproduced saturation effect.
  arXiv  Detail & Related papers  (2026-06-02T23:54:27Z)
• GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectives [48.545980031973556]
  GAMBIT is a benchmark with three evaluation modes and two independent scores for evaluating imposter detectors.<n>The benchmark comes with a dataset of 27,804 labeled instances spanning 240 co-evolved imposter strategies.
  arXiv  Detail & Related papers  (2026-05-09T16:07:23Z)
• MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents [2.1942030377331245]
  Coding agents often pass per-prompt safety review yet ship exploitable code when their tasks are decomposed into routine engineering tickets.<n>We introduce MOSAIC-Bench, a benchmark of 199 three-stage attack chains paired with deterministic exploit oracles on deployed software substrates.<n>We show that nine production coding agents compose innocuous tickets at 53-86% end-to-end ASR with only two refusals across all staged runs.
  arXiv  Detail & Related papers  (2026-05-05T16:38:23Z)
• ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence [0.0]
  ARC-AGI-3 is an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments.<n>We present the benchmark design, its efficiency-based scoring framework grounded in human action baselines, and the methodology used to construct, validate, and calibrate the environments.
  arXiv  Detail & Related papers  (2026-03-24T21:58:16Z)
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists [71.02281792867531]
  We evaluate two refinement modes: guided refinement and self-refinement.<n>In guided refinement, both proprietary LMs and large open-weight LMs can leverage targeted feedback to refine responses to near-perfect levels within five turns.<n>These findings suggest that frontier LMs require breakthroughs to self-refine their incorrect responses.
  arXiv  Detail & Related papers  (2025-11-27T07:20:52Z)
• Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People [81.63702981397408]
  Given limited resources, to what extent do agents based on language models (LMs) act rationally?<n>We develop methods to benchmark and enhance agentic information-seeking, drawing on insights from human behavior.<n>For Spotter agents, our approach boosts accuracy by up to 14.7% absolute over LM-only baselines; for Captain agents, it raises expected information gain (EIG) by up to 0.227 bits (94.2% of the achievable noise ceiling)
  arXiv  Detail & Related papers  (2025-10-23T17:57:28Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Efficient Policy Space Response Oracles [61.71849698253696]
  Policy Space Response Oracle method (PSRO) provides a general solution to Nash equilibrium in two-player zero-sum games. Central to our development is the newly-introduced of minimax optimization on unrestricted-restricted (URR) games. We report a 50x speedup in wall-time, 10x data efficiency, and similar exploitability as existing PSRO methods on Kuhn and Leduc Poker games.
  arXiv  Detail & Related papers  (2022-01-28T17:54:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.