Complex Locomotion Skill Learning via Differentiable Physics
- URL: http://arxiv.org/abs/2206.02341v2
- Date: Sat, 28 Oct 2023 11:01:53 GMT
- Title: Complex Locomotion Skill Learning via Differentiable Physics
- Authors: Yu Fang and Jiancheng Liu and Mingrui Zhang and Jiasheng Zhang and
Yidong Ma and Minchen Li and Yuanming Hu and Chenfanfu Jiang and Tiantian Liu
- Abstract summary: Differentiable physics enables efficient-based optimizations of neural network (NN) controllers.
We present a practical learning framework that outputs unified NN controllers capable of tasks with significantly improved complexity and diversity.
- Score: 30.868690308658174
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Differentiable physics enables efficient gradient-based optimizations of
neural network (NN) controllers. However, existing work typically only delivers
NN controllers with limited capability and generalizability. We present a
practical learning framework that outputs unified NN controllers capable of
tasks with significantly improved complexity and diversity. To systematically
improve training robustness and efficiency, we investigated a suite of
improvements over the baseline approach, including periodic activation
functions, and tailored loss functions. In addition, we find our adoption of
batching and an Adam optimizer effective in training complex locomotion tasks.
We evaluate our framework on differentiable mass-spring and material point
method (MPM) simulations, with challenging locomotion tasks and multiple robot
designs. Experiments show that our learning framework, based on differentiable
physics, delivers better results than reinforcement learning and converges much
faster. We demonstrate that users can interactively control soft robot
locomotion and switch among multiple goals with specified velocity, height, and
direction instructions using a unified NN controller trained in our system.
Code is available at
https://github.com/erizmr/Complex-locomotion-skill-learning-via-differentiable-physics.
Related papers
- Multi-Objective Algorithms for Learning Open-Ended Robotic Problems [1.0124625066746598]
Quadrupedal locomotion is a complex, open-ended problem vital to expanding autonomous vehicle reach.
Traditional reinforcement learning approaches often fall short due to training instability and sample inefficiency.
We propose a novel method leveraging multi-objective evolutionary algorithms as an automatic curriculum learning mechanism.
arXiv Detail & Related papers (2024-11-11T16:26:42Z) - Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotion [2.7052274816160966]
We propose a novel reinforcement learning framework for training neural network controllers for complex robotic systems consisting of both rewards and constraints.
The learning framework is applied to train controllers for several legged robots with different morphology and physical attributes to traverse challenging terrains.
arXiv Detail & Related papers (2023-08-24T03:06:20Z) - Learning and Adapting Agile Locomotion Skills by Transferring Experience [71.8926510772552]
We propose a framework for training complex robotic skills by transferring experience from existing controllers to jumpstart learning new tasks.
We show that our method enables learning complex agile jumping behaviors, navigating to goal locations while walking on hind legs, and adapting to new environments.
arXiv Detail & Related papers (2023-04-19T17:37:54Z) - Hindsight States: Blending Sim and Real Task Elements for Efficient
Reinforcement Learning [61.3506230781327]
In robotics, one approach to generate training data builds on simulations based on dynamics models derived from first principles.
Here, we leverage the imbalance in complexity of the dynamics to learn more sample-efficiently.
We validate our method on several challenging simulated tasks and demonstrate that it improves learning both alone and when combined with an existing hindsight algorithm.
arXiv Detail & Related papers (2023-03-03T21:55:04Z) - Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance
Action Space [7.116986445066885]
Reinforcement Learning has led to promising results on a range of challenging decision-making tasks.
Fast human-like adaptive control methods can optimize complex robotic interactions, yet fail to integrate multimodal feedback needed for unstructured tasks.
We propose to factor the learning problem in a hierarchical learning and adaption architecture to get the best of both worlds.
arXiv Detail & Related papers (2021-10-19T12:09:02Z) - Efficient Differentiable Simulation of Articulated Bodies [89.64118042429287]
We present a method for efficient differentiable simulation of articulated bodies.
This enables integration of articulated body dynamics into deep learning frameworks.
We show that reinforcement learning with articulated systems can be accelerated using gradients provided by our method.
arXiv Detail & Related papers (2021-09-16T04:48:13Z) - PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable
Physics [89.81550748680245]
We introduce a new differentiable physics benchmark called PasticineLab.
In each task, the agent uses manipulators to deform the plasticine into the desired configuration.
We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark.
arXiv Detail & Related papers (2021-04-07T17:59:23Z) - Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces.
We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space.
NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
arXiv Detail & Related papers (2020-12-04T18:59:32Z) - Learning Agile Robotic Locomotion Skills by Imitating Animals [72.36395376558984]
Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics.
We present an imitation learning system that enables legged robots to learn agile locomotion skills by imitating real-world animals.
arXiv Detail & Related papers (2020-04-02T02:56:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.