Look Before You Leap: Autonomous Exploration for LLM Agents

• URL: http://arxiv.org/abs/2605.16143v1
• Date: Fri, 15 May 2026 16:24:16 GMT
• Title: Look Before You Leap: Autonomous Exploration for LLM Agents
• Abstract summary: Large language model based agents often fail in unfamiliar environments due to premature exploitation.<n>We identify autonomous exploration as a critical yet underexplored capability for building adaptive agents.
• Score: 46.001025916022066
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large language model based agents often fail in unfamiliar environments due to premature exploitation: a tendency to act on prior knowledge before acquiring sufficient environment-specific information. We identify autonomous exploration as a critical yet underexplored capability for building adaptive agents. To formalize and quantify this capability, we introduce Exploration Checkpoint Coverage, a verifiable metric that measures how broadly an agent discovers key states, objects, and affordances. Our systematic evaluation reveals that agents trained with standard task-oriented reinforcement learning consistently exhibit narrow and repetitive behaviors that impede downstream performance. To address this limitation, we develop a training strategy that interleaves task-execution rollouts and exploration rollouts, with each type of rollout optimized by its corresponding verifiable reward. Building on this training strategy, we propose the Explore-then-Act paradigm, which decouples information-gathering from task execution: agents first utilize an interaction budget to acquire grounded environmental knowledge, then leverage it for task resolution. Our results demonstrate that learning to systematically explore is imperative for building generalizable and real-world-ready agents.

Related papers

• Learning to Retrieve from Agent Trajectories [72.8923565916533]
  We argue that retrieval models for agentic search should be trained directly from agent interaction data.<n>We introduce learning to retrieve from agent trajectories as a new training paradigm, where supervision is derived from multi-step agent interactions.<n>Our results highlight agent trajectories as a practical and scalable supervision source, pointing to a promising direction for retrieval in the era of agentic search.
  arXiv  Detail & Related papers  (2026-03-30T17:59:02Z)
• Temporal Representations for Exploration: Learning Complex Exploratory Behavior without Extrinsic Rewards [39.328230174948025]
  We propose an exploration method that leverages temporal contrastive representations to guide exploration.<n>We demonstrate that such representations can enable the learning of complex exploratory x in locomotion, manipulation, and embodied-AI tasks.
  arXiv  Detail & Related papers  (2026-03-02T15:55:27Z)
• Unsupervised Learning of Efficient Exploration: Pre-training Adaptive Policies via Self-Imposed Goals [0.0]
  Unsupervised pre-training can equip reinforcement learning agents with prior knowledge and accelerate learning in downstream tasks.<n>We present ULEE, an unsupervised meta-learning method that combines an in-context learner with an adversarial goal-generation strategy.
  arXiv  Detail & Related papers  (2026-01-27T17:10:29Z)
• Scaling Environments for LLM Agents in the Era of Learning from Interaction: A Survey [30.673419015614233]
  A growing consensus is that agents should interact directly with environments and learn from experience through reinforcement learning.<n>We formalize this iterative process as the Generation-Execution-Feedback (GEF) loop, where environments generate tasks to challenge agents, return observations in response to agents' actions during task execution, and provide evaluative feedback on rollouts for subsequent learning.<n>Under this paradigm, environments function as indispensable producers of experiential data, highlighting the need to scale them toward greater complexity, realism, and interactivity.
  arXiv  Detail & Related papers  (2025-11-12T12:56:25Z)
• Agent Learning via Early Experience [93.83579011718858]
  A long-term goal of language agents is to learn and improve through their own experience, ultimately outperforming humans in complex, real-world tasks.<n>Most current agents rely on supervised fine-tuning on expert data, which is challenging to scale and generalizes poorly.<n>We study two strategies of using such data: (1) Implicit world modeling, which uses collected states to ground the policy in environment dynamics; and (2) Self-reflection, where the agent learns from its suboptimal actions to improve reasoning and decision-making.
  arXiv  Detail & Related papers  (2025-10-09T17:59:17Z)
• Curriculum-Based Multi-Tier Semantic Exploration via Deep Reinforcement Learning [1.8374319565577155]
  This paper presents a novel Deep Reinforcement Learning architecture that is specifically designed for resource efficient semantic exploration.<n>A key methodological contribution is the integration of a Vision-Language Model (VLM) common-sense through a layered reward function.<n>We show that our agent achieves significantly enhanced object discovery rates and develops a learned capability to effectively navigate towards semantically rich regions.
  arXiv  Detail & Related papers  (2025-09-11T11:10:08Z)
• Agentic Knowledgeable Self-awareness [79.25908923383776]
  KnowSelf is a data-centric approach that applies agents with knowledgeable self-awareness like humans.<n>Our experiments demonstrate that KnowSelf can outperform various strong baselines on different tasks and models with minimal use of external knowledge.
  arXiv  Detail & Related papers  (2025-04-04T16:03:38Z)
• Reinforcement Learning with Prototypical Representations [114.35801511501639]
  Proto-RL is a self-supervised framework that ties representation learning with exploration through prototypical representations. These prototypes simultaneously serve as a summarization of the exploratory experience of an agent as well as a basis for representing observations. This enables state-of-the-art downstream policy learning on a set of difficult continuous control tasks.
  arXiv  Detail & Related papers  (2021-02-22T18:56:34Z)
• Planning to Explore via Self-Supervised World Models [120.31359262226758]
  Plan2Explore is a self-supervised reinforcement learning agent. We present a new approach to self-supervised exploration and fast adaptation to new tasks. Without any training supervision or task-specific interaction, Plan2Explore outperforms prior self-supervised exploration methods.
  arXiv  Detail & Related papers  (2020-05-12T17:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.