ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence

• URL: http://arxiv.org/abs/2603.24621v1
• Date: Tue, 24 Mar 2026 21:58:16 GMT
• Title: ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence
• Abstract summary: ARC-AGI-3 is an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments.<n>We present the benchmark design, its efficiency-based scoring framework grounded in human action baselines, and the methodology used to construct, validate, and calibrate the environments.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents must explore, infer goals, build internal models of environment dynamics, and plan effective action sequences without explicit instructions. Like its predecessors ARC-AGI-1 and 2, ARC-AGI-3 focuses entirely on evaluating fluid adaptive efficiency on novel tasks, while avoiding language and external knowledge. ARC-AGI-3 environments only leverage Core Knowledge priors and are difficulty-calibrated via extensive testing with human test-takers. Our testing shows humans can solve 100% of the environments, in contrast to frontier AI systems which, as of March 2026, score below 1%. In this paper, we present the benchmark design, its efficiency-based scoring framework grounded in human action baselines, and the methodology used to construct, validate, and calibrate the environments.

Related papers

• Apodex Discovery: Reality Benchmarks and Environments for Evaluating and Building Discoverative Artificial Intelligence [86.61561123749011]
  We introduce Apodex Discovery, a framework for building and evaluating discoverative AI through the heavy-duty solver.<n>First, a problem-scouting process surveyed 561 industries across 16 sectors, assembled 423 high-value real-world problems, and selected 20 for the initial release.<n>Second, a common environment-task-episode abstraction provides data, tools, constraints, feedback, trajectory recording, and verification of intermediate artifacts and final submissions.
  arXiv  Detail & Related papers  (2026-08-11T18:48:40Z)
• Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3 [0.0]
  We investigate all 25 public ARC-AGI-3 games and find that every one is reachable through non-intelligent strategies.<n>This benchmark critique implies the public evaluation set cannot discriminate intelligent exploration.<n>We present AERA, a three-phase (EXPLORE / VERIFY / PLAN) agent achieving RHAE=0.2116 on these 25 games.
  arXiv  Detail & Related papers  (2026-05-25T15:13:18Z)
• Stargazer: A Scalable Model-Fitting Benchmark Environment for AI Agents under Astrophysical Constraints [48.80158223838359]
  We introduce Stargazer, a scalable environment for evaluating AI agents on dynamic, iterative physics-grounded model-fitting tasks.<n>Stargazer comprises 120 tasks across three difficulty tiers, including 20 real archival cases.<n>Our evaluation of eight frontier agents reveals a gap between numerical optimization and adherence to physical constraints.
  arXiv  Detail & Related papers  (2026-04-17T03:38:50Z)
• The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning [5.870990919877351]
  The Abstraction and Reasoning Corpus (ARC-AGI) has become a key benchmark for fluid intelligence in AI.<n>This survey presents the first cross-generation analysis of 82 approaches across three benchmark versions and the ARC Prize 2024-2025 competitions.
  arXiv  Detail & Related papers  (2026-03-09T23:16:19Z)
• Autonomous Continual Learning of Computer-Use Agents for Environment Adaptation [57.65688895630163]
  We introduce ACuRL, an Autonomous Curriculum Reinforcement Learning framework that continually adapts agents to specific environments with zero human data.<n>Our method effectively enables both intra-environment and cross-environment continual learning, yielding 4-22% performance gains without forgetting existing environments.
  arXiv  Detail & Related papers  (2026-02-10T23:06:02Z)
• ARC Prize 2025: Technical Report [0.45671221781968335]
  ARC-AGI benchmark series serves as a critical measure of few-shot generalization on novel tasks.<n>The ARC Prize 2025 global competition targeted the newly released ARC-AGI-2 dataset.<n>The defining theme of 2025 is the emergence of the refinement loop.
  arXiv  Detail & Related papers  (2026-01-15T23:23:56Z)
• Graph-Based Exploration for ARC-AGI-3 Interactive Reasoning Tasks [0.32116198597240847]
  We present a training-free approach for solving interactive reasoning tasks in the ARC-AGI-3 benchmark.<n>Our method combines vision-based frame processing with systematic state-space exploration using graph-structured representations.<n>It solves a median of 30 out of 52 levels across six games and ranks 3rd on the private leaderboard.
  arXiv  Detail & Related papers  (2025-12-30T11:40:23Z)
• Vector Symbolic Algebras for the Abstraction and Reasoning Corpus [1.2031796234206134]
  ARC-AGI is a generative, few-shot fluid intelligence benchmark.<n>We propose a cognitively plausible ARC-AGI solver.<n>Our solver integrates System 1 intuitions with System 2 reasoning.
  arXiv  Detail & Related papers  (2025-11-11T20:07:50Z)
• ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems [0.03431023404301193]
  ARC-AGI-2 preserves the input-output pair task format of its predecessor, ensuring continuity for researchers.<n>It incorporates a newly curated and expanded set of tasks specifically designed to assess abstract reasoning and problem-solving abilities.<n> ARC-AGI-2 aims to serve as a next-generation tool for rigorously measuring progress towards more general and human-like AI capabilities.
  arXiv  Detail & Related papers  (2025-05-17T04:34:48Z)
• Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answering [87.76784654371312]
  Embodied Question Answering requires agents to dynamically explore 3D environments, actively gather visual information, and perform multi-step reasoning to answer questions.<n>Existing datasets often introduce biases or prior knowledge, leading to disembodied reasoning.<n>We construct the largest dataset designed specifically to evaluate both exploration and reasoning capabilities.
  arXiv  Detail & Related papers  (2025-03-14T06:29:47Z)
• Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge [62.981429762309226]
  The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually. We propose to improve phrase grounding models' ability on localizing the active objects by: learning the role of objects undergoing change and extracting them accurately from the instructions. We evaluate our framework on Ego4D and Epic-Kitchens datasets.
  arXiv  Detail & Related papers  (2023-10-23T16:14:05Z)
• OpenAGI: When LLM Meets Domain Experts [51.86179657467822]
  Human Intelligence (HI) excels at combining basic skills to solve complex tasks. This capability is vital for Artificial Intelligence (AI) and should be embedded in comprehensive AI Agents. We introduce OpenAGI, an open-source platform designed for solving multi-step, real-world tasks.
  arXiv  Detail & Related papers  (2023-04-10T03:55:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.