AGI Maze as a Benchmark Framework for World-Modeling Agents

• URL: http://arxiv.org/abs/2607.00627v1
• Date: Wed, 01 Jul 2026 08:43:08 GMT
• Title: AGI Maze as a Benchmark Framework for World-Modeling Agents
• Abstract summary: Large language models (LLMs) are powerful pattern-completion systems, but their default operating mode does not reliably produce persistent, manipulable representations of an external world.<n>AGI Maze is a lightweight framework for building such environments without requiring high-dimensional sensory inputs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are powerful pattern-completion systems, but their default operating mode - predicting the next token from a static context - does not reliably produce persistent, manipulable representations of an external world. Many tasks that look like "reasoning" in text become substantially harder once the environment is partially observable, stateful, and requires memory and structured hypotheses about hidden state. AGI Maze is a lightweight framework for building such environments without requiring high-dimensional sensory inputs. It provides a family of grid-based maze tasks with a clean API and multiple difficulty regimes. The goal is to create benchmarks where agents must learn and use world state representations, not just infer a local rule over readily provided observations. We provide an initial evaluation of several vanilla LLMs on simple mazes showing that they fail to represent mazes internally at LLM inference time. We also introduce a baseline agent, which is allowed to use its message history as a working memory to construct descriptions of observations at agentic runtime. Although this can improve performance, it is still insufficient for an LLM agent to reliably solve even small mazes within a step budget that is more than enough for humans.

Related papers

• Bridging the Agent-World Gap: Text World Models for LLM-based Agents [83.8237017249529]
  Large language model (LLM)-based agents are increasingly used in interactive textual environments.<n>Many remain largely reactive, mapping observations to actions without an explicit model of how these environments are structured and evolve.<n>This motivates text world models (TWMs): transition models over textual states that, given a state and a candidate action, predict the resulting webpage, terminal output, API response, or user reply.<n>We systematically review text world models for LLM-based agents, organized around a formal framework and the agent lifecycle.
  arXiv  Detail & Related papers  (2026-06-08T04:58:52Z)
• GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0) [56.81743709880371]
  Long-horizon large language model (LLM) agents are fundamentally limited by context.<n>We argue that long-horizon performance is determined not by context length, but by how much decision-relevant information is maintained within a finite context budget.<n>We present GenericAgent (GA), a general-purpose, self-evolving LLM agent system built around a single principle: information density and context.
  arXiv  Detail & Related papers  (2026-04-18T17:59:15Z)
• Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Models [73.89069781682032]
  We quantify modality preference of Native Omni-modal Large Language Models (OLLMs)<n>Unlike the text-dominance'' of traditional VLMs, most OLLMs exhibit a pronounced visual preference.<n>Our work provides both a mechanistic understanding and a practical tool for building more trustworthy OLLMs.
  arXiv  Detail & Related papers  (2026-04-18T08:25:52Z)
• Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs [100.02824137397464]
  We investigate how Large Language Models adapt their internal representations when encountering inputs of increasing difficulty.<n>We reveal a consistent and quantifiable phenomenon: as task difficulty increases, the last hidden states of LLMs become substantially sparser.<n>This sparsity--difficulty relation is observable across diverse models and domains.
  arXiv  Detail & Related papers  (2026-03-03T18:48:15Z)
• Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions [18.455501447828343]
  Spatial Intelligence (SI) has predominantly relied on Vision-Language Models (VLMs)<n>We introduce SiT-Bench, a novel benchmark designed to evaluate the SI performance of Large Language Models (LLMs) without pixel-level input.<n>We find that explicit spatial reasoning significantly boosts performance, suggesting that LLMs possess latent world-modeling potential.
  arXiv  Detail & Related papers  (2026-01-07T05:13:52Z)
• WRAVAL -- WRiting Assist eVALuation [7.441391098440092]
  Small Language Models (SLMs) typically score 3-4 times lower than Large Language Models (LLMs) on reasoning metrics.<n>We propose an evaluation framework specifically designed to highlight SLMs' capabilities in non-reasoning tasks.
  arXiv  Detail & Related papers  (2025-12-19T09:21:27Z)
• Where MLLMs Attend and What They Rely On: Explaining Autoregressive Token Generation [59.40886078302025]
  Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in aligning visual inputs with natural language outputs.<n>Yet, the extent to which generated tokens depend on visual modalities remains poorly understood.<n>We present a lightweight black-box framework for explaining autoregressive token generation in MLLMs.
  arXiv  Detail & Related papers  (2025-09-26T15:38:42Z)
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction [80.88654868264645]
  Arranged and Organized Extraction Benchmark designed to evaluate ability of large language models to comprehend fragmented documents.<n>AOE includes 11 carefully crafted tasks across three diverse domains, requiring models to generate context-specific schema tailored to varied input queries.<n>Results show that even the most advanced models struggled significantly.
  arXiv  Detail & Related papers  (2025-07-22T06:37:51Z)
• Foundation Models for Remote Sensing: An Analysis of MLLMs for Object Localization [7.0683335354070085]
  We analyze recent MLLMs that have been explicitly trained to include fine-grained spatial reasoning capabilities.<n>We demonstrate that these models are performant in certain settings, making them well suited for zero-shot scenarios.
  arXiv  Detail & Related papers  (2025-04-14T21:34:06Z)
• AI-native Memory: A Pathway from LLMs Towards AGI [25.19572633670963]
  Large language models (LLMs) have demonstrated the world with the sparks of artificial general intelligence (AGI) We envision a pathway from LLMs to AGI through the integration of emphmemory. As an intermediate stage, the memory will likely be in the form of natural language descriptions.
  arXiv  Detail & Related papers  (2024-06-26T12:51:37Z)
• Statler: State-Maintaining Language Models for Embodied Reasoning [19.884696137429813]
  We propose Statler, a framework in which large language models are prompted to maintain an estimate of the world state. Our framework then conditions each action on the estimate of the current world state. It significantly outperforms strong competing methods on several robot planning tasks.
  arXiv  Detail & Related papers  (2023-06-30T17:58:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.