Reference Grounded Skill Discovery

• URL: http://arxiv.org/abs/2510.06203v1
• Date: Tue, 07 Oct 2025 17:55:01 GMT
• Title: Reference Grounded Skill Discovery
• Authors: Seungeun Rho, Aaron Trinh, Danfei Xu, Sehoon Ha,
• Abstract summary: We present Reference-Grounded Skill Discovery (RGSD), a novel algorithm that grounds skill discovery in a semantically meaningful latent space.<n>On a simulated SMPL humanoid with 359-D observations and 69-D actions, RGSD learns structured skills including walking, running, punching, and side stepping.<n>Our results suggest that lightweight reference-guided grounding offers a practical path to discovering semantically rich and structured skills in high-DoF systems.
• Score: 13.23914921356941
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling unsupervised skill discovery algorithms to high-DoF agents remains challenging. As dimensionality increases, the exploration space grows exponentially, while the manifold of meaningful skills remains limited. Therefore, semantic meaningfulness becomes essential to effectively guide exploration in high-dimensional spaces. In this work, we present Reference-Grounded Skill Discovery (RGSD), a novel algorithm that grounds skill discovery in a semantically meaningful latent space using reference data. RGSD first performs contrastive pretraining to embed motions on a unit hypersphere, clustering each reference trajectory into a distinct direction. This grounding enables skill discovery to simultaneously involve both imitation of reference behaviors and the discovery of semantically related diverse behaviors. On a simulated SMPL humanoid with 359-D observations and 69-D actions, RGSD learns structured skills including walking, running, punching, and side stepping, and also discovers related novel behaviors. In downstream control tasks, RGSD outperforms imitation-based skill acquisition baselines. Our results suggest that lightweight reference-guided grounding offers a practical path to discovering semantically rich and structured skills in high-DoF systems.

Related papers

• Task-Aware Exploration via a Predictive Bisimulation Metric [13.445649480300132]
  We present TEB, a Task-aware Exploration approach that tightly couples task-relevant representations with exploration through a predictive Bisimulation metric.<n>Specifically, TEB leverages the metric not only to learn behaviorally grounded task representations but also to measure behaviorally intrinsic novelty over the learned latent space.<n>Building on this robust metric, we design potential-based exploration bonuses, which measure the relative novelty of adjacent observations over the latent space.
  arXiv  Detail & Related papers  (2026-02-21T05:30:34Z)
• SUSD: Structured Unsupervised Skill Discovery through State Factorization [12.57032768854794]
  Unsupervised Skill Discovery (USD) aims to autonomously learn a diverse set of skills without relying on extrinsic rewards.<n>We introduce SUSD, a novel framework that harnesses the compositional structure of environments by factorizing the state space into independent components.<n>SUSD allocates distinct skill variables to different factors, enabling more fine-grained control on the skill discovery process.
  arXiv  Detail & Related papers  (2026-02-02T04:21:33Z)
• Scalable Exploration for High-Dimensional Continuous Control via Value-Guided Flow [12.08071565601655]
  We introduce Q-guided Flow Exploration (Qflex), a scalable reinforcement learning method that conducts exploration directly in the native high-dimensional action space.<n>During training, Qflex traverses actions from a learnable source distribution along a probability flow induced by the learned value function, aligning exploration with task-relevant gradients rather than isotropic noise.<n>Our results indicate that value-guided flows offer a principled and practical route to exploration at scale.
  arXiv  Detail & Related papers  (2026-01-27T15:30:10Z)
• SpatialActor: Exploring Disentangled Spatial Representations for Robust Robotic Manipulation [63.48859753472547]
  SpatialActor is a framework for robust robotic manipulation that explicitly decouples semantics and geometry.<n>It achieves state-of-the-art performance with 87.4% on RLBench and improves by 13.9% to 19.4% under varying noisy conditions.
  arXiv  Detail & Related papers  (2025-11-12T18:59:08Z)
• From reactive to cognitive: brain-inspired spatial intelligence for embodied agents [50.99942960312313]
  Brain-inspired Spatial Cognition for Navigation (BSC-Nav) is a unified framework for constructing and leveraging structured spatial memory in embodied agents.<n> BSC-Nav builds allocentric cognitive maps from egocentric trajectories and contextual cues, and dynamically retrieves spatial knowledge aligned with semantic goals.
  arXiv  Detail & Related papers  (2025-08-24T03:20:48Z)
• Unsupervised Skill Discovery through Skill Regions Differentiation [6.088346462603191]
  Unsupervised Reinforcement Learning (RL) aims to discover diverse behaviors that can accelerate the learning of downstream tasks.<n>We propose a novel skill discovery objective that maximizes the deviation of the state density of one skill from the explored regions of other skills.<n>We also formulate an intrinsic reward based on the learned autoencoder that resembles count-based exploration in a compact latent space.
  arXiv  Detail & Related papers  (2025-06-17T11:30:04Z)
• Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models [81.08295968057453]
  We present IVE, an agentic exploration framework inspired by human curiosity.<n>We evaluate IVE in both simulated and real-world tabletop environments.
  arXiv  Detail & Related papers  (2025-05-12T17:59:11Z)
• Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation [69.01029651113386]
  Embodied-RAG is a framework that enhances the model of an embodied agent with a non-parametric memory system.<n>At its core, Embodied-RAG's memory is structured as a semantic forest, storing language descriptions at varying levels of detail.<n>We demonstrate that Embodied-RAG effectively bridges RAG to the robotics domain, successfully handling over 250 explanation and navigation queries.
  arXiv  Detail & Related papers  (2024-09-26T21:44:11Z)
• CODEX: A Cluster-Based Method for Explainable Reinforcement Learning [0.0]
  We present a method that incorporates semantic clustering, which can effectively summarize RL agent behavior in the state-action space. Experiments on the MiniGrid and StarCraft II gaming environments reveal the semantic clusters retain temporal as well as entity information.
  arXiv  Detail & Related papers  (2023-12-07T11:04:37Z)
• METRA: Scalable Unsupervised RL with Metric-Aware Abstraction [69.90741082762646]
  Metric-Aware Abstraction (METRA) is a novel unsupervised reinforcement learning objective. By learning to move in every direction in the latent space, METRA obtains a tractable set of diverse behaviors. We show that METRA can discover a variety of useful behaviors even in complex, pixel-based environments.
  arXiv  Detail & Related papers  (2023-10-13T06:43:11Z)
• Learning Representative Trajectories of Dynamical Systems via Domain-Adaptive Imitation [0.0]
  We propose DATI, a deep reinforcement learning agent designed for domain-adaptive trajectory imitation. Our experiments show that DATI outperforms baseline methods for imitation learning and optimal control in this setting. Its generalization to a real-world scenario is shown through the discovery of abnormal motion patterns in maritime traffic.
  arXiv  Detail & Related papers  (2023-04-19T15:53:48Z)
• Hierarchical Skills for Efficient Exploration [70.62309286348057]
  In reinforcement learning, pre-trained low-level skills have the potential to greatly facilitate exploration. Prior knowledge of the downstream task is required to strike the right balance between generality (fine-grained control) and specificity (faster learning) in skill design. We propose a hierarchical skill learning framework that acquires skills of varying complexity in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-10-20T22:29:32Z)
• Learning Task Agnostic Skills with Data-driven Guidance [0.0]
  This paper proposes a framework for guiding the skill discovery towards the subset of expert-visited states. We apply our method in various reinforcement learning tasks and show that such a projection results in more useful behaviours.
  arXiv  Detail & Related papers  (2021-08-04T06:53:10Z)
• Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigation [11.868792440783055]
  We show that learning to estimate metrics quantifying the spatial relationships between an agent at a given location and a goal to reach has a high positive impact in Multi-Object Navigation settings. A learning-based agent from the literature trained with the proposed auxiliary losses was the winning entry to the Multi-Object Navigation Challenge.
  arXiv  Detail & Related papers  (2021-07-13T12:01:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.