Explore Beyond the Boundary Using Entropic Information

• URL: http://arxiv.org/abs/2607.29419v1
• Date: Fri, 31 Jul 2026 13:41:51 GMT
• Title: Explore Beyond the Boundary Using Entropic Information
• Abstract summary: In reinforcement learning, exploration with sparse and delayed rewards presents a significant challenge.<n>We propose Entropic Information for Exploration (ENTINEX), a novel method that enhances exploration by incentivizing agents to explore beyond the boundaries of the state distribution.
• Score: 7.589048964013273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In reinforcement learning, exploration with sparse and delayed rewards presents a significant challenge due to the limited feedback available for guiding the learning process. Addressing this issue requires extensive exploration in the state space to discover valuable reward signals. In this paper, we propose Entropic Information for Exploration (ENTINEX), a novel method that enhances exploration by incentivizing agents to explore beyond the boundaries of the state distribution. ENTINEX achieves this by assigning intrinsic rewards to these boundaries, leveraging entropic information to identify them effectively. Through extensive experimentation, we demonstrate that ENTINEX consistently improves exploration performance in environments characterized by sparse and delayed rewards. Our experimental results show that ENTINEX outperforms existing exploration methods, highlighting its effectiveness in both sparse and delayed reward scenarios.

Related papers

• Clearing the Fog: Towards Installing and Refining Proactive Exploration Capabilities in LLM Agents [108.55958466397932]
  ours is a novel method designed to instill and refine proactive exploration.<n>ours consists of two components: (1) Exploratory Data Construction, which synthesizes exploration-rich trajectories to mitigate the hindsight bias of standard demonstrations; and (2) RL Optimization with Contrastive Signal Guidance, which leverages contrastive trajectory pairs to distinguish productive exploration from redundant wandering.
  arXiv  Detail & Related papers  (2026-08-14T14:24:50Z)
• DiPO: Disentangled Perplexity Policy Optimization for Fine-grained Exploration-Exploitation Trade-Off [87.58233482504308]
  Reinforcement Learning with Verifiable Rewards (RLVR) has catalyzed significant advances in the reasoning capabilities of Large Language Models (LLMs)<n>This paper fully analyze the exploration and exploitation dilemma of extremely hard and easy samples during the training and propose a new fine-grained trade-off mechanism.
  arXiv  Detail & Related papers  (2026-04-15T14:12:10Z)
• Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration [77.36486933055907]
  We propose a novel imitation learning algorithm called Imitation Learning with Double Exploration (ILDE)<n>ILDE implements exploration in two aspects: (1) optimistic policy optimization via an exploration bonus that rewards state-action pairs with high uncertainty to potentially improve the convergence to the expert policy, and (2) curiosity-driven exploration of the states that deviate from the demonstration trajectories to potentially yield beyond-expert performance.<n> Empirically, we demonstrate that ILDE outperforms the state-of-the-art imitation learning algorithms in terms of sample efficiency and achieves beyond-expert performance on Atari and MuJoCo tasks with fewer demonstrations than in previous work.
  arXiv  Detail & Related papers  (2025-06-25T10:39:32Z)
• Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answering [87.76784654371312]
  Embodied Question Answering requires agents to dynamically explore 3D environments, actively gather visual information, and perform multi-step reasoning to answer questions.<n>Existing datasets often introduce biases or prior knowledge, leading to disembodied reasoning.<n>We construct the largest dataset designed specifically to evaluate both exploration and reasoning capabilities.
  arXiv  Detail & Related papers  (2025-03-14T06:29:47Z)
• MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization [91.80034860399677]
  Reinforcement learning algorithms aim to balance exploiting the current best strategy with exploring new options that could lead to higher rewards.<n>We introduce a framework, MaxInfoRL, for balancing intrinsic and extrinsic exploration.<n>We show that our approach achieves sublinear regret in the simplified setting of multi-armed bandits.
  arXiv  Detail & Related papers  (2024-12-16T18:59:53Z)
• Information Content Exploration [1.7034813545878589]
  We propose a new intrinsic reward that systemically quantifies exploratory behavior and promotes state coverage. We show that our information theoretic reward induces efficient exploration and outperforms in various games.
  arXiv  Detail & Related papers  (2023-10-10T16:51:32Z)
• Never Explore Repeatedly in Multi-Agent Reinforcement Learning [40.35950679063337]
  We propose a dynamic reward scaling approach to combat "revisitation" We show enhanced performance in demanding environments like Google Research Football and StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2023-08-19T05:27:48Z)
• Successor-Predecessor Intrinsic Exploration [18.440869985362998]
  We focus on exploration with intrinsic rewards, where the agent transiently augments the external rewards with self-generated intrinsic rewards. We propose Successor-Predecessor Intrinsic Exploration (SPIE), an exploration algorithm based on a novel intrinsic reward combining prospective and retrospective information. We show that SPIE yields more efficient and ethologically plausible exploratory behaviour in environments with sparse rewards and bottleneck states than competing methods.
  arXiv  Detail & Related papers  (2023-05-24T16:02:51Z)
• Rewarding Episodic Visitation Discrepancy for Exploration in Reinforcement Learning [64.8463574294237]
  We propose Rewarding Episodic Visitation Discrepancy (REVD) as an efficient and quantified exploration method. REVD provides intrinsic rewards by evaluating the R'enyi divergence-based visitation discrepancy between episodes. It is tested on PyBullet Robotics Environments and Atari games.
  arXiv  Detail & Related papers  (2022-09-19T08:42:46Z)
• Perturbation-based exploration methods in deep reinforcement learning [0.0]
  We investigate the effect of perturbations in policy and reward spaces on the exploratory behavior of the agent. We show that simple acts of perturbing the policy just before the softmax layer and introduction of sporadic reward bonuses into the domain can greatly enhance exploration.
  arXiv  Detail & Related papers  (2020-11-10T22:57:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.