Task-Aware Exploration via a Predictive Bisimulation Metric

• URL: http://arxiv.org/abs/2602.18724v1
• Date: Sat, 21 Feb 2026 05:30:34 GMT
• Title: Task-Aware Exploration via a Predictive Bisimulation Metric
• Abstract summary: We present TEB, a Task-aware Exploration approach that tightly couples task-relevant representations with exploration through a predictive Bisimulation metric.<n>Specifically, TEB leverages the metric not only to learn behaviorally grounded task representations but also to measure behaviorally intrinsic novelty over the learned latent space.<n>Building on this robust metric, we design potential-based exploration bonuses, which measure the relative novelty of adjacent observations over the latent space.
• Score: 13.445649480300132
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accelerating exploration in visual reinforcement learning under sparse rewards remains challenging due to the substantial task-irrelevant variations. Despite advances in intrinsic exploration, many methods either assume access to low-dimensional states or lack task-aware exploration strategies, thereby rendering them fragile in visual domains. To bridge this gap, we present TEB, a Task-aware Exploration approach that tightly couples task-relevant representations with exploration through a predictive Bisimulation metric. Specifically, TEB leverages the metric not only to learn behaviorally grounded task representations but also to measure behaviorally intrinsic novelty over the learned latent space. To realize this, we first theoretically mitigate the representation collapse of degenerate bisimulation metrics under sparse rewards by internally introducing a simple but effective predicted reward differential. Building on this robust metric, we design potential-based exploration bonuses, which measure the relative novelty of adjacent observations over the latent space. Extensive experiments on MetaWorld and Maze2D show that TEB achieves superior exploration ability and outperforms recent baselines.

Related papers

• Towards Generalizable Visually Grounded Exploration of Household Devices [74.66117923766632]
  Key bottleneck of realizing general autonomous embodied agents lies in Generalizable Visually Grounded Exploration.<n>We introduce VGEBench, a benchmark designed to evaluate the generalizable visually grounded exploration capabilities of Vision-Language Models.<n>Unlike static datasets, we construct a Logic-Driven State Machine framework. This framework simulates multi-turn interaction loops, compelling agents to achieve goals by active visual perception and feedback-driven correction.
  arXiv  Detail & Related papers  (2026-09-01T07:42:45Z)
• Beyond Thinking: Imagining in 360$^\circ$ for Humanoid Visual Search [50.04233606482412]
  Humanoid Visual Search (HVS) requires agents to actively explore immersive 360$circ$ environments.<n>We propose Imagining in 360$circ$, a novel framework that decouples the exploration process into a specialized Imaginator and an Actor.<n>We show that explicitly modeling semantic spatial priors drastically improves search efficiency and success rates in complex, in-the-wild environments.
  arXiv  Detail & Related papers  (2026-05-09T20:10:53Z)
• Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration [12.531650952835493]
  In this paper, we propose a new paradigm that explicitly separates exploration from exploitation and bypasses RL during the exploration phase.<n>By removing the overhead of policy optimization, our approach explores an order of magnitude more efficiently than standard intrinsic motivation baselines on hard Atari benchmarks.<n>We demonstrate that the discovered trajectories can be distilled into deployable policies using existing supervised backward learning algorithms.
  arXiv  Detail & Related papers  (2026-03-23T17:56:52Z)
• Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Exploration [12.928422281441968]
  Embodied tasks typically require agents to actively explore unknown environments and reason about the scene to achieve a specific goal.<n>When deployed in real life, agents often face sequential tasks, where each new sub-task follows the completion of the previous one.<n>We introduce SEER-Bench, a new Sequential Embodied Exploration and Reasoning Benchmark encompassing two classic embodied tasks.<n>We propose 3DSPMR, a 3D SPatial Memory Reasoning approach that exploits relational, visual, and geometric cues from explored regions.
  arXiv  Detail & Related papers  (2025-12-02T06:35:30Z)
• Reference Grounded Skill Discovery [13.23914921356941]
  We present Reference-Grounded Skill Discovery (RGSD), a novel algorithm that grounds skill discovery in a semantically meaningful latent space.<n>On a simulated SMPL humanoid with 359-D observations and 69-D actions, RGSD learns structured skills including walking, running, punching, and side stepping.<n>Our results suggest that lightweight reference-guided grounding offers a practical path to discovering semantically rich and structured skills in high-DoF systems.
  arXiv  Detail & Related papers  (2025-10-07T17:55:01Z)
• Unsupervised Skill Discovery through Skill Regions Differentiation [6.088346462603191]
  Unsupervised Reinforcement Learning (RL) aims to discover diverse behaviors that can accelerate the learning of downstream tasks.<n>We propose a novel skill discovery objective that maximizes the deviation of the state density of one skill from the explored regions of other skills.<n>We also formulate an intrinsic reward based on the learned autoencoder that resembles count-based exploration in a compact latent space.
  arXiv  Detail & Related papers  (2025-06-17T11:30:04Z)
• Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models [81.08295968057453]
  We present IVE, an agentic exploration framework inspired by human curiosity.<n>We evaluate IVE in both simulated and real-world tabletop environments.
  arXiv  Detail & Related papers  (2025-05-12T17:59:11Z)
• Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answering [87.76784654371312]
  Embodied Question Answering requires agents to dynamically explore 3D environments, actively gather visual information, and perform multi-step reasoning to answer questions.<n>Existing datasets often introduce biases or prior knowledge, leading to disembodied reasoning.<n>We construct the largest dataset designed specifically to evaluate both exploration and reasoning capabilities.
  arXiv  Detail & Related papers  (2025-03-14T06:29:47Z)
• Disentangling Exploration of Large Language Models by Optimal Exploitation [17.346054308224993]
  This work isolates exploration as the sole objective, tasking an agent with gathering information that enhances future returns.<n>We decompose missing rewards into their exploration and exploitation components based on the optimal achievable return.
  arXiv  Detail & Related papers  (2025-01-15T16:30:29Z)
• Never Explore Repeatedly in Multi-Agent Reinforcement Learning [40.35950679063337]
  We propose a dynamic reward scaling approach to combat "revisitation" We show enhanced performance in demanding environments like Google Research Football and StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2023-08-19T05:27:48Z)
• Rewarding Episodic Visitation Discrepancy for Exploration in Reinforcement Learning [64.8463574294237]
  We propose Rewarding Episodic Visitation Discrepancy (REVD) as an efficient and quantified exploration method. REVD provides intrinsic rewards by evaluating the R'enyi divergence-based visitation discrepancy between episodes. It is tested on PyBullet Robotics Environments and Atari games.
  arXiv  Detail & Related papers  (2022-09-19T08:42:46Z)
• Seeing Differently, Acting Similarly: Imitation Learning with Heterogeneous Observations [126.78199124026398]
  In many real-world imitation learning tasks, the demonstrator and the learner have to act in different but full observation spaces. In this work, we model the above learning problem as Heterogeneous Observations Learning (HOIL) We propose the Importance Weighting with REjection (IWRE) algorithm based on the techniques of importance-weighting, learning with rejection, and active querying to solve the key challenge of occupancy measure matching.
  arXiv  Detail & Related papers  (2021-06-17T05:44:04Z)
• Batch Exploration with Examples for Scalable Robotic Reinforcement Learning [63.552788688544254]
  Batch Exploration with Examples (BEE) explores relevant regions of the state-space guided by a modest number of human provided images of important states. BEE is able to tackle challenging vision-based manipulation tasks both in simulation and on a real Franka robot.
  arXiv  Detail & Related papers  (2020-10-22T17:49:25Z)
• Planning to Explore via Self-Supervised World Models [120.31359262226758]
  Plan2Explore is a self-supervised reinforcement learning agent. We present a new approach to self-supervised exploration and fast adaptation to new tasks. Without any training supervision or task-specific interaction, Plan2Explore outperforms prior self-supervised exploration methods.
  arXiv  Detail & Related papers  (2020-05-12T17:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.