Unsupervised Hierarchical Skill Discovery

• URL: http://arxiv.org/abs/2601.23156v1
• Date: Fri, 30 Jan 2026 16:41:13 GMT
• Title: Unsupervised Hierarchical Skill Discovery
• Authors: Damion Harvey, Geraud Nangue Tasse, Branden Ingram, Benjamin Rosman, Steven James,
• Abstract summary: We consider the problem of unsupervised skill segmentation and hierarchical structure discovery in reinforcement learning.<n>We propose a method that segments unlabelled trajectories into skills and induces a hierarchical structure over them using a grammar-based approach.<n>We evaluate our approach in high-dimensional, pixel-based environments, including Craftax and the full, unmodified version of Minecraft.
• Score: 11.230382111014073
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of unsupervised skill segmentation and hierarchical structure discovery in reinforcement learning. While recent approaches have sought to segment trajectories into reusable skills or options, most rely on action labels, rewards, or handcrafted annotations, limiting their applicability. We propose a method that segments unlabelled trajectories into skills and induces a hierarchical structure over them using a grammar-based approach. The resulting hierarchy captures both low-level behaviours and their composition into higher-level skills. We evaluate our approach in high-dimensional, pixel-based environments, including Craftax and the full, unmodified version of Minecraft. Using metrics for skill segmentation, reuse, and hierarchy quality, we find that our method consistently produces more structured and semantically meaningful hierarchies than existing baselines. Furthermore, as a proof of concept for utility, we demonstrate that these discovered hierarchies accelerate and stabilise learning on downstream reinforcement learning tasks.

Related papers

• Provable Learning of Random Hierarchy Models and Hierarchical Shallow-to-Deep Chaining [58.69016084278948]
  We consider a hierarchical context-free grammar introduced by arXiv:2307.02129 and conjectured to separate deep and shallow networks.<n>We prove that, under mild conditions, a deep convolutional network can be efficiently trained to learn this function class.
  arXiv  Detail & Related papers  (2026-01-27T16:19:54Z)
• Hierarchical Semantic Tree Anchoring for CLIP-Based Class-Incremental Learning [11.82771798674077]
  Class-Incremental Learning (CIL) enables models to learn new classes continually while preserving past knowledge.<n>But real-world visual and linguistic concepts are inherently hierarchical.<n>We propose HASTEN that anchors hierarchical information into CIL to reduce catastrophic forgetting.
  arXiv  Detail & Related papers  (2025-11-19T17:14:47Z)
• A Semantics-Aware Hierarchical Self-Supervised Approach to Classification of Remote Sensing Images [12.282079123411947]
  We present a novel Semantics-Aware Hierarchical Consensus (SAHC) method for learning hierarchical features and relationships.<n>The SAHC method is evaluated on three benchmark datasets with different degrees of hierarchical complexity.<n> Experimental results show both the effectiveness of the proposed approach in guiding network learning and the robustness of the hierarchical consensus for remote sensing image classification tasks.
  arXiv  Detail & Related papers  (2025-10-06T15:30:39Z)
• Learning and Evaluating Hierarchical Feature Representations [3.770103075126785]
  We propose a novel framework, Hierarchical Composition of Orthogonal Subspaces (Hier-COS)<n>Hier-COS learns to map deep feature embeddings into a vector space that is, by design, consistent with the structure of a given taxonomy tree.<n>We demonstrate that Hier-COS achieves state-of-the-art hierarchical performance across all the datasets while simultaneously beating top-1 accuracy in all but one case.
  arXiv  Detail & Related papers  (2025-03-10T20:59:41Z)
• Hierarchical Action Recognition: A Contrastive Video-Language Approach with Hierarchical Interactions [19.741453194665276]
  We formalize the novel task of hierarchical video recognition, and propose a video-language learning framework tailored for hierarchical recognition. Specifically, our framework encodes dependencies between hierarchical category levels, and applies a top-down constraint to filter recognition predictions. We demonstrate the efficacy of our approach for hierarchical recognition, significantly outperforming conventional methods.
  arXiv  Detail & Related papers  (2024-05-28T01:17:22Z)
• Reinforcement Learning with Options and State Representation [105.82346211739433]
  This thesis aims to explore the reinforcement learning field and build on existing methods to produce improved ones. It addresses such goals by decomposing learning tasks in a hierarchical fashion known as Hierarchical Reinforcement Learning.
  arXiv  Detail & Related papers  (2024-03-16T08:30:55Z)
• Creating Multi-Level Skill Hierarchies in Reinforcement Learning [0.0]
  We propose an answer based on a graphical representation of how the interaction between an agent and its environment may unfold. Our approach uses modularity maximisation as a central organising principle to expose the structure of the interaction graph at multiple levels of abstraction.
  arXiv  Detail & Related papers  (2023-06-16T17:23:49Z)
• Weakly-supervised Action Localization via Hierarchical Mining [76.00021423700497]
  Weakly-supervised action localization aims to localize and classify action instances in the given videos temporally with only video-level categorical labels. We propose a hierarchical mining strategy under video-level and snippet-level manners, i.e., hierarchical supervision and hierarchical consistency mining. We show that HiM-Net outperforms existing methods on THUMOS14 and ActivityNet1.3 datasets with large margins by hierarchically mining the supervision and consistency.
  arXiv  Detail & Related papers  (2022-06-22T12:19:09Z)
• Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework [75.79736930414715]
  We present a hierarchical multi-label representation learning framework that can leverage all available labels and preserve the hierarchical relationship between classes. We introduce novel hierarchy preserving losses, which jointly apply a hierarchical penalty to the contrastive loss, and enforce the hierarchy constraint.
  arXiv  Detail & Related papers  (2022-04-27T21:41:44Z)
• Hierarchical Skills for Efficient Exploration [70.62309286348057]
  In reinforcement learning, pre-trained low-level skills have the potential to greatly facilitate exploration. Prior knowledge of the downstream task is required to strike the right balance between generality (fine-grained control) and specificity (faster learning) in skill design. We propose a hierarchical skill learning framework that acquires skills of varying complexity in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-10-20T22:29:32Z)
• Provable Hierarchy-Based Meta-Reinforcement Learning [50.17896588738377]
  We analyze HRL in the meta-RL setting, where learner learns latent hierarchical structure during meta-training for use in a downstream task. We provide "diversity conditions" which, together with a tractable optimism-based algorithm, guarantee sample-efficient recovery of this natural hierarchy. Our bounds incorporate common notions in HRL literature such as temporal and state/action abstractions, suggesting that our setting and analysis capture important features of HRL in practice.
  arXiv  Detail & Related papers  (2021-10-18T17:56:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.