Information-Based Exploration via Random Features for Reinforcement Learning

• URL: http://arxiv.org/abs/2607.17981v1
• Date: Mon, 20 Jul 2026 14:15:03 GMT
• Title: Information-Based Exploration via Random Features for Reinforcement Learning
• Abstract summary: We introduce Random Feature Information Gain (RFIG), which uses random features to approximate information gain and compute exploration bonuses in non-countable spaces.<n>RFIG achieves competitive performance with well-established deep exploration methods while offering superior theoretical interpretation.
• Score: 6.896797484250302
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Representation learning has enabled classical exploration strategies to be extended to deep Reinforcement Learning (RL), but often makes algorithms more complex and theoretical guarantees harder to establish. We introduce Random Feature Information Gain (RFIG), grounded in Bayesian kernel methods theory, which uses random Fourier features to approximate information gain and compute exploration bonuses in non-countable spaces. We provide error bounds on information gain approximation and avoid the black-box aspects of neural network-based uncertainty estimation, for optimism-based exploration. We present practical details that make RFIG scalable to deep RL scenarios, enabling smooth integration into standard deep RL algorithms. Experimental evaluation across diverse control and navigation tasks demonstrates that RFIG achieves competitive performance with well-established deep exploration methods while offering superior theoretical interpretation.

Related papers

• Sample-Efficient Neurosymbolic Deep Reinforcement Learning [49.60927398960061]
  We propose a neuro-symbolic Deep RL approach that integrates background symbolic knowledge to improve sample efficiency.<n>Online reasoning is performed to guide the training process through two mechanisms.<n>We show improved performance over a state-of-the-art reward machine baseline.
  arXiv  Detail & Related papers  (2026-01-06T09:28:53Z)
• Value of Information-Enhanced Exploration in Bootstrapped DQN [2.6173443955754903]
  In this paper, we integrate the notion of (expected) value of information (EVOI) within the well-known Bootstrapped DQN algorithmic framework.<n>Specifically, we develop two novel algorithms that incorporate the expected gain from learning the value of information into Bootstrapped DQN.<n>Our experiments in complex, sparse-reward Atari games demonstrate increased performance, all the while making better use of uncertainty.
  arXiv  Detail & Related papers  (2025-11-04T20:22:58Z)
• On Efficient Bayesian Exploration in Model-Based Reinforcement Learning [0.24578723416255752]
  We address the challenge of data-efficient exploration in reinforcement learning by examining existing principled, information-theoretic approaches to intrinsic motivation.<n>We prove that exploration bonuses naturally signal epistemic information gains and converge to zero once the agent becomes sufficiently certain about the environment's dynamics and rewards.<n>We then outline a general framework - Predictive Trajectory Sampling with Bayesian Exploration (PTS-BE) - which integrates model-based planning with information-theoretic bonuses to achieve sample-efficient deep exploration.
  arXiv  Detail & Related papers  (2025-07-03T14:03:47Z)
• MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization [91.80034860399677]
  Reinforcement learning algorithms aim to balance exploiting the current best strategy with exploring new options that could lead to higher rewards.<n>We introduce a framework, MaxInfoRL, for balancing intrinsic and extrinsic exploration.<n>We show that our approach achieves sublinear regret in the simplified setting of multi-armed bandits.
  arXiv  Detail & Related papers  (2024-12-16T18:59:53Z)
• Random Latent Exploration for Deep Reinforcement Learning [71.88709402926415]
  We introduce Random Latent Exploration (RLE), a simple yet effective exploration strategy in reinforcement learning (RL)<n>On average, RLE outperforms noise-based methods, which perturb the agent's actions, and bonus-based exploration, which rewards the agent for attempting novel behaviors.<n>RLE is as simple as noise-based methods, as it avoids complex bonus calculations but retains the deep exploration benefits of bonus-based methods.
  arXiv  Detail & Related papers  (2024-07-18T17:55:22Z)
• How does Your RL Agent Explore? An Optimal Transport Analysis of Occupancy Measure Trajectories [8.429001045596687]
  We represent the learning process of an RL algorithm as a sequence of policies generated during training. We then study the policy trajectory induced in the manifold of state-action occupancy measures.
  arXiv  Detail & Related papers  (2024-02-14T11:55:50Z)
• On the Importance of Exploration for Generalization in Reinforcement Learning [89.63074327328765]
  We propose EDE: Exploration via Distributional Ensemble, a method that encourages exploration of states with high uncertainty. Our algorithm is the first value-based approach to achieve state-of-the-art on both Procgen and Crafter.
  arXiv  Detail & Related papers  (2023-06-08T18:07:02Z)
• Provable Reward-Agnostic Preference-Based Reinforcement Learning [61.39541986848391]
  Preference-based Reinforcement Learning (PbRL) is a paradigm in which an RL agent learns to optimize a task using pair-wise preference-based feedback over trajectories. We propose a theoretical reward-agnostic PbRL framework where exploratory trajectories that enable accurate learning of hidden reward functions are acquired.
  arXiv  Detail & Related papers  (2023-05-29T15:00:09Z)
• STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning [111.75423966239092]
  We propose an exploration incentive in terms of the integral probability metric (IPM) between a current estimate of the transition model and the unknown optimal. Based on KSD, we develop a novel algorithm algo: textbfSTEin information dirtextbfEcted exploration for model-based textbfReinforcement LearntextbfING.
  arXiv  Detail & Related papers  (2023-01-28T00:49:28Z)
• A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges [51.699348215510575]
  Reinforcement Learning (RL) is a popular machine learning paradigm where intelligent agents interact with the environment to fulfill a long-term goal.<n>Despite the encouraging results achieved, the deep neural network-based backbone is widely deemed as a black box that impedes practitioners to trust and employ trained agents in realistic scenarios where high security and reliability are essential.<n>To alleviate this issue, a large volume of literature devoted to shedding light on the inner workings of the intelligent agents has been proposed, by constructing intrinsic interpretability or post-hoc explainability.
  arXiv  Detail & Related papers  (2022-11-12T13:52:06Z)
• Reinforcement Learning with Probabilistically Complete Exploration [27.785017885906313]
  We propose Rapidly Randomly-exploring Reinforcement Learning (R3L) We formulate exploration as a search problem and leverage widely-used planning algorithms to find initial solutions. We experimentally demonstrate the method, requiring only a fraction of exploration samples and achieving better performance.
  arXiv  Detail & Related papers  (2020-01-20T02:11:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.