COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space

• URL: http://arxiv.org/abs/2606.00950v1
• Date: Sun, 31 May 2026 02:04:35 GMT
• Title: COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space
• Abstract summary: Unsupervised skill discovery (USD) aims to learn diverse behaviors without reward functions, but often results in task-irrelevant or hazardous behaviors.<n>We propose COLLIE, a framework that leverages dense unsupervised data to construct a semantically coherent skill latent space.<n>Our experiments show that COLLIE learns diverse, human-aligned skills, avoids hazardous behaviors, and achieves superior downstream performance with minimal human feedback.
• Score: 15.241002193763087
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Unsupervised skill discovery (USD) aims to learn diverse behaviors without reward functions, but often results in task-irrelevant or hazardous behaviors due to uniform exploration. Guided skill discovery (GSD) addresses this issue by incorporating human intent to focus exploration on meaningful regions. However, existing GSD methods typically require training additional guidance models, and rely on pre-defined rules or expert demonstration, which can be ineffective under sparse, online-collected human feedback. To overcome this, we propose COLLIE, a GSD framework that leverages dense unsupervised data to construct a semantically coherent skill latent space. This latent space is well-structured, enabling reliable guidance with sparse online feedback. Moreover, its semantic coherence property enables training-free construction of guidance signals, eliminating the need for additional model training beyond skill learning. Theoretical analysis justifies the effectiveness of our training-free guidance signal, while experiments across diverse state-based and pixel-based tasks show that COLLIE learns diverse, human-aligned skills, avoids hazardous behaviors, and achieves superior downstream performance with minimal human feedback.

Related papers

• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• Path-level Hindsight Instructions for Semantic Exploration in Vision-Language Navigation [49.96273402030084]
  We introduce Phi-Nav, a unified on-policy framework that leverages hindsight reasoning to align instructions with the agent's actual exploratory journey.<n>Phi-Nav bridges the critical semantic supervision gap inherent in on-policy methods, transforming semantically unlabeled movement into dense training signals.
  arXiv  Detail & Related papers  (2026-07-02T06:11:07Z)
• Leveraging Human Feedback for Semantically-Relevant Skill Discovery [11.543807097834785]
  We propose Semantically Relevant Skill Discovery (SRSD) as a novel human-in-the-loop approach to skill discovery.<n>SRSD collects semantic labels from human feedback and learns a reward function to encourage skills to be more semantically diverse and relevant.<n>We demonstrate that SRSD can improve semantic diversity and discover relevant behaviours while scaling effectively to a large variety of behaviours.
  arXiv  Detail & Related papers  (2026-04-27T07:28:28Z)
• Reference Grounded Skill Discovery [13.23914921356941]
  We present Reference-Grounded Skill Discovery (RGSD), a novel algorithm that grounds skill discovery in a semantically meaningful latent space.<n>On a simulated SMPL humanoid with 359-D observations and 69-D actions, RGSD learns structured skills including walking, running, punching, and side stepping.<n>Our results suggest that lightweight reference-guided grounding offers a practical path to discovering semantically rich and structured skills in high-DoF systems.
  arXiv  Detail & Related papers  (2025-10-07T17:55:01Z)
• Behavioral Exploration: Learning to Explore via In-Context Adaptation [53.92981562916783]
  We train a long-context generative model to predict expert actions conditioned on a context of past observations and a measure of how exploratory'' the expert's behaviors are relative to this context.<n>This enables the model to not only mimic the behavior of an expert, but also, by feeding its past history of interactions into its context, to select different expert behaviors than what have been previously selected.<n>We demonstrate the effectiveness of our method in both simulated locomotion and manipulation settings, as well as on real-world robotic manipulation tasks.
  arXiv  Detail & Related papers  (2025-07-11T21:36:19Z)
• Human-Aligned Skill Discovery: Balancing Behaviour Exploration and Alignment [14.948610521764415]
  We propose Human-aligned Skill Discovery (HaSD) to discover safer, more aligned skills.<n>HaSD simultaneously optimises skill diversity and alignment with human values.<n>We demonstrate its effectiveness in both 2D navigation and SafetyGymnasium environments.
  arXiv  Detail & Related papers  (2025-01-29T06:14:27Z)
• Unsupervised Continual Anomaly Detection with Contrastively-learned Prompt [80.43623986759691]
  We introduce a novel Unsupervised Continual Anomaly Detection framework called UCAD. The framework equips the UAD with continual learning capability through contrastively-learned prompts. We conduct comprehensive experiments and set the benchmark on unsupervised continual anomaly detection and segmentation.
  arXiv  Detail & Related papers  (2024-01-02T03:37:11Z)
• Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learning [66.00600682711995]
  Human object interaction (HOI) detection plays a crucial role in human-centric scene understanding and serves as a fundamental building-block for many vision tasks. One generalizable and scalable strategy for HOI detection is to use weak supervision, learning from image-level annotations only. This is inherently challenging due to ambiguous human-object associations, large search space of detecting HOIs and highly noisy training signal. We develop a CLIP-guided HOI representation capable of incorporating the prior knowledge at both image level and HOI instance level, and adopt a self-taught mechanism to prune incorrect human-object associations.
  arXiv  Detail & Related papers  (2023-03-02T14:41:31Z)
• Imitation from Observation With Bootstrapped Contrastive Learning [12.048166025000976]
  Imitation from observation (IfO) is a learning paradigm that consists of training autonomous agents in a Markov Decision Process. We present BootIfOL, an IfO algorithm that aims to learn a reward function that takes an agent trajectory and compares it to an expert. We evaluate our approach on a variety of control tasks showing that we can train effective policies using a limited number of demonstrative trajectories.
  arXiv  Detail & Related papers  (2023-02-13T17:32:17Z)
• Autonomous Reinforcement Learning: Formalism and Benchmarking [106.25788536376007]
  Real-world embodied learning, such as that performed by humans and animals, is situated in a continual, non-episodic world. Common benchmark tasks in RL are episodic, with the environment resetting between trials to provide the agent with multiple attempts. This discrepancy presents a major challenge when attempting to take RL algorithms developed for episodic simulated environments and run them on real-world platforms.
  arXiv  Detail & Related papers  (2021-12-17T16:28:06Z)
• Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reaching [98.25207998996066]
  We build on the mutual information framework for skill discovery and introduce UPSIDE to address the coverage-directedness trade-off. We illustrate in several navigation and control environments how the skills learned by UPSIDE solve sparse-reward downstream tasks better than existing baselines.
  arXiv  Detail & Related papers  (2021-10-27T14:22:19Z)
• Learning Task Agnostic Skills with Data-driven Guidance [0.0]
  This paper proposes a framework for guiding the skill discovery towards the subset of expert-visited states. We apply our method in various reinforcement learning tasks and show that such a projection results in more useful behaviours.
  arXiv  Detail & Related papers  (2021-08-04T06:53:10Z)
• Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control [67.23580984118479]
  Imitation Learning (IL) is an effective framework to learn visuomotor skills from offline demonstration data. Hand-eye Action Networks (HAN) can approximate human's hand-eye coordination behaviors by learning from human teleoperated demonstrations.
  arXiv  Detail & Related papers  (2021-02-28T01:49:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.