Diverse Skill Discovery for Quadruped Robots via Unsupervised Learning

• URL: http://arxiv.org/abs/2602.09767v1
• Date: Tue, 10 Feb 2026 13:28:13 GMT
• Title: Diverse Skill Discovery for Quadruped Robots via Unsupervised Learning
• Abstract summary: Unsupervised skill discovery can potentially reduce burdens by learning a diverse repertoire of useful skills driven by intrinsic motivation.<n>We introduce an Orthogonal Mixture-of-Experts architecture that prevents diverse behaviors from collapsing into overlapping representations.<n>We also design a multi-discriminator framework in which different discriminators operate on distinct observation spaces, effectively mitigating reward hacking.
• Score: 2.9196984708596716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning necessitates meticulous reward shaping by specialists to elicit target behaviors, while imitation learning relies on costly task-specific data. In contrast, unsupervised skill discovery can potentially reduce these burdens by learning a diverse repertoire of useful skills driven by intrinsic motivation. However, existing methods exhibit two key limitations: they typically rely on a single policy to master a versatile repertoire of behaviors without modeling the shared structure or distinctions among them, which results in low learning efficiency; moreover, they are susceptible to reward hacking, where the reward signal increases and converges rapidly while the learned skills display insufficient actual diversity. In this work, we introduce an Orthogonal Mixture-of-Experts (OMoE) architecture that prevents diverse behaviors from collapsing into overlapping representations, enabling a single policy to master a wide spectrum of locomotion skills. In addition, we design a multi-discriminator framework in which different discriminators operate on distinct observation spaces, effectively mitigating reward hacking. We evaluated our method on the 12-DOF Unitree A1 quadruped robot, demonstrating a diverse set of locomotion skills. Our experiments demonstrate that the proposed framework boosts training efficiency and yields an 18.3\% expansion in state-space coverage compared to the baseline.

Related papers

• SUSD: Structured Unsupervised Skill Discovery through State Factorization [12.57032768854794]
  Unsupervised Skill Discovery (USD) aims to autonomously learn a diverse set of skills without relying on extrinsic rewards.<n>We introduce SUSD, a novel framework that harnesses the compositional structure of environments by factorizing the state space into independent components.<n>SUSD allocates distinct skill variables to different factors, enabling more fine-grained control on the skill discovery process.
  arXiv  Detail & Related papers  (2026-02-02T04:21:33Z)
• CORE: Code-based Inverse Self-Training Framework with Graph Expansion for Virtual Agents [69.88668127604875]
  We present CORE, a Code-based Inverse Self-Training Framework with Graph Expansion.<n> CORE bridges imitation and exploration, offering a novel training framework that promotes behavioral diversity.<n> Experiments on Web and Android platforms demonstrate that CORE significantly improves both overall performance and generalization.
  arXiv  Detail & Related papers  (2026-01-05T15:24:05Z)
• Diversifying Policy Behaviors with Extrinsic Behavioral Curiosity [27.272921087408164]
  This work introduces Quality Diversity Inverse Reinforcement Learning (QD-IRL) and Extrinsic Behavioral Curiosity (EBC)<n>QD-IRL integrates quality-diversity optimization with IRL methods, enabling agents to learn diverse behaviors from limited demonstrations.<n>EBC allows agents to receive additional curiosity rewards from an external critic based on how novel the behaviors are.
  arXiv  Detail & Related papers  (2024-10-08T15:49:33Z)
• RILe: Reinforced Imitation Learning [60.63173816209543]
  RILe (Reinforced Learning) is a framework that combines the strengths of imitation learning and inverse reinforcement learning to learn a dense reward function efficiently.<n>Our framework produces high-performing policies in high-dimensional tasks where direct imitation fails to replicate complex behaviors.
  arXiv  Detail & Related papers  (2024-06-12T17:56:31Z)
• SLIM: Skill Learning with Multiple Critics [8.645929825516818]
  Self-supervised skill learning aims to acquire useful behaviors that leverage the underlying dynamics of the environment. Latent variable models, based on mutual information, have been successful in this task but still struggle in the context of robotic manipulation. We introduce SLIM, a multi-critic learning approach for skill discovery with a particular focus on robotic manipulation.
  arXiv  Detail & Related papers  (2024-02-01T18:07:33Z)
• ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discovery [12.277005054008017]
  We propose textbfContrastive dynatextbfmic textbfSkill textbfDiscovery textbf(ComSD).<n>ComSD generates diverse and exploratory unsupervised skills through a novel intrinsic incentive, named contrastive dynamic reward.<n>It can also discover distinguishable and far-reaching exploration skills in the challenging tree-like 2D maze.
  arXiv  Detail & Related papers  (2023-09-29T12:53:41Z)
• Generating Personas for Games with Multimodal Adversarial Imitation Learning [47.70823327747952]
  Reinforcement learning has been widely successful in producing agents capable of playing games at a human level. Going beyond reinforcement learning is necessary to model a wide range of human playstyles. This paper presents a novel imitation learning approach to generate multiple persona policies for playtesting.
  arXiv  Detail & Related papers  (2023-08-15T06:58:19Z)
• Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motions [19.626042478612572]
  We propose a cooperative adversarial method for obtaining versatile policies with controllable skill sets from unlabeled datasets. We show that by utilizing unsupervised skill discovery in the generative imitation learning framework, novel and useful skills emerge with successful task fulfillment. Finally, the obtained versatile policies are tested on an agile quadruped robot called Solo 8 and present faithful replications of diverse skills encoded in the demonstrations.
  arXiv  Detail & Related papers  (2022-09-16T12:49:04Z)
• Discovering Generalizable Skills via Automated Generation of Diverse Tasks [82.16392072211337]
  We propose a method to discover generalizable skills via automated generation of a diverse set of tasks. As opposed to prior work on unsupervised discovery of skills, our method pairs each skill with a unique task produced by a trainable task generator. A task discriminator defined on the robot behaviors in the generated tasks is jointly trained to estimate the evidence lower bound of the diversity objective. The learned skills can then be composed in a hierarchical reinforcement learning algorithm to solve unseen target tasks.
  arXiv  Detail & Related papers  (2021-06-26T03:41:51Z)
• Bridging the Imitation Gap by Adaptive Insubordination [88.35564081175642]
  We show that when the teaching agent makes decisions with access to privileged information, this information is marginalized during imitation learning. We propose 'Adaptive Insubordination' (ADVISOR) to address this gap. ADVISOR dynamically weights imitation and reward-based reinforcement learning losses during training, enabling on-the-fly switching between imitation and exploration.
  arXiv  Detail & Related papers  (2020-07-23T17:59:57Z)
• Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning [81.12201426668894]
  We develop efficient reinforcement learning methods that acquire diverse skills without any reward function, and then repurpose these skills for downstream tasks. We show that our proposed algorithm provides substantial improvement in learning efficiency, making reward-free real-world training feasible. We also demonstrate that the learned skills can be composed using model predictive control for goal-oriented navigation, without any additional training.
  arXiv  Detail & Related papers  (2020-04-27T17:38:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.