ModSkill: Physical Character Skill Modularization

• URL: http://arxiv.org/abs/2502.14140v1
• Date: Wed, 19 Feb 2025 22:55:49 GMT
• Title: ModSkill: Physical Character Skill Modularization
• Authors: Yiming Huang, Zhiyang Dou, Lingjie Liu,
• Abstract summary: We introduce a novel skill learning framework, ModSkill, that decouples complex full-body skills into compositional, modular skills for independent body parts. Our results show that this modularized skill learning framework, enhanced by generative sampling, outperforms existing methods in precise full-body motion tracking.
• Score: 21.33764810227885
• License:
• Abstract: Human motion is highly diverse and dynamic, posing challenges for imitation learning algorithms that aim to generalize motor skills for controlling simulated characters. Previous methods typically rely on a universal full-body controller for tracking reference motion (tracking-based model) or a unified full-body skill embedding space (skill embedding). However, these approaches often struggle to generalize and scale to larger motion datasets. In this work, we introduce a novel skill learning framework, ModSkill, that decouples complex full-body skills into compositional, modular skills for independent body parts. Our framework features a skill modularization attention layer that processes policy observations into modular skill embeddings that guide low-level controllers for each body part. We also propose an Active Skill Learning approach with Generative Adaptive Sampling, using large motion generation models to adaptively enhance policy learning in challenging tracking scenarios. Our results show that this modularized skill learning framework, enhanced by generative sampling, outperforms existing methods in precise full-body motion tracking and enables reusable skill embeddings for diverse goal-driven tasks.

Related papers

• Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputs [16.41735119504929]
  This work focuses on generating realistic, physically-based human behaviors from multi-modal inputs, which may only partially specify the desired motion. The input may come from a VR controller providing arm motion and body velocity, partial key-point animation, computer vision applied to videos, or even higher-level motion goals. We introduce the Masked Humanoid Controller (MHC), a novel approach that applies multi-objective imitation learning on augmented and selectively masked motion demonstrations.
  arXiv  Detail & Related papers  (2025-02-08T17:02:11Z)
• Integrating Controllable Motion Skills from Demonstrations [30.943279225315308]
  We introduce a flexible multi-skill integration framework named Controllable Skills Integration (CSI) CSI enables the integration of a diverse set of motion skills with varying styles into a single policy without the need for complex reward tuning. Our experiments demonstrate that CSI can flexibly integrate a diverse array of motion skills more comprehensively and facilitate the transitions between different skills.
  arXiv  Detail & Related papers  (2024-08-06T08:01:02Z)
• Large Motion Model for Unified Multi-Modal Motion Generation [50.56268006354396]
  Large Motion Model (LMM) is a motion-centric, multi-modal framework that unifies mainstream motion generation tasks into a generalist model. LMM tackles these challenges from three principled aspects.
  arXiv  Detail & Related papers  (2024-04-01T17:55:11Z)
• Modular Neural Network Policies for Learning In-Flight Object Catching with a Robot Hand-Arm System [55.94648383147838]
  We present a modular framework designed to enable a robot hand-arm system to learn how to catch flying objects. Our framework consists of five core modules: (i) an object state estimator that learns object trajectory prediction, (ii) a catching pose quality network that learns to score and rank object poses for catching, (iii) a reaching control policy trained to move the robot hand to pre-catch poses, and (iv) a grasping control policy trained to perform soft catching motions. We conduct extensive evaluations of our framework in simulation for each module and the integrated system, to demonstrate high success rates of in-flight
  arXiv  Detail & Related papers  (2023-12-21T16:20:12Z)
• Learning and Adapting Agile Locomotion Skills by Transferring Experience [71.8926510772552]
  We propose a framework for training complex robotic skills by transferring experience from existing controllers to jumpstart learning new tasks. We show that our method enables learning complex agile jumping behaviors, navigating to goal locations while walking on hind legs, and adapting to new environments.
  arXiv  Detail & Related papers  (2023-04-19T17:37:54Z)
• ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters [123.88692739360457]
  General-purpose motor skills enable humans to perform complex tasks. These skills also provide powerful priors for guiding their behaviors when learning new tasks. We present a framework for learning versatile and reusable skill embeddings for physically simulated characters.
  arXiv  Detail & Related papers  (2022-05-04T06:13:28Z)
• Learning Multi-Objective Curricula for Deep Reinforcement Learning [55.27879754113767]
  Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of deep reinforcement learning (DRL) In this paper, we propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula. In addition to existing hand-designed curricula paradigms, we further design a flexible memory mechanism to learn an abstract curriculum.
  arXiv  Detail & Related papers  (2021-10-06T19:30:25Z)
• Learning compositional models of robot skills for task and motion planning [39.36562555272779]
  We learn to use sensorimotor primitives to solve complex long-horizon manipulation problems. We use state-of-the-art methods for active learning and sampling. We evaluate our approach both in simulation and in the real world through measuring the quality of the selected primitive actions.
  arXiv  Detail & Related papers  (2020-06-08T20:45:34Z)
• Learning Agile Robotic Locomotion Skills by Imitating Animals [72.36395376558984]
  Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics. We present an imitation learning system that enables legged robots to learn agile locomotion skills by imitating real-world animals.
  arXiv  Detail & Related papers  (2020-04-02T02:56:16Z)
• Learning Whole-body Motor Skills for Humanoids [25.443880385966114]
  This paper presents a hierarchical framework for Deep Reinforcement Learning that acquires motor skills for a variety of push recovery and balancing behaviors. The policy is trained in a physics simulator with realistic setting of robot model and low-level impedance control that are easy to transfer the learned skills to real robots.
  arXiv  Detail & Related papers  (2020-02-07T19:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.