ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse
Quadruped Robots
- URL: http://arxiv.org/abs/2310.10486v2
- Date: Fri, 8 Mar 2024 14:10:02 GMT
- Title: ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse
Quadruped Robots
- Authors: Milad Shafiee, Guillaume Bellegarda and Auke Ijspeert
- Abstract summary: We show that drawing inspiration from animal motor control allows us to effectively train a single locomotion policy for quadruped robots.
Our policy modulates a representation of the Central Pattern Generator (CPG) in the spinal cord.
We observe robust performance, even when adding a 15 kg load, equivalent to 125% of the A1 robot's nominal mass.
- Score: 4.557963624437784
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Learning a locomotion policy for quadruped robots has traditionally been
constrained to a specific robot morphology, mass, and size. The learning
process must usually be repeated for every new robot, where hyperparameters and
reward function weights must be re-tuned to maximize performance for each new
system. Alternatively, attempting to train a single policy to accommodate
different robot sizes, while maintaining the same degrees of freedom (DoF) and
morphology, requires either complex learning frameworks, or mass, inertia, and
dimension randomization, which leads to prolonged training periods. In our
study, we show that drawing inspiration from animal motor control allows us to
effectively train a single locomotion policy capable of controlling a diverse
range of quadruped robots. The robot differences encompass: a variable number
of DoFs, (i.e. 12 or 16 joints), three distinct morphologies, a broad mass
range spanning from 2 kg to 200 kg, and nominal standing heights ranging from
18 cm to 100 cm. Our policy modulates a representation of the Central Pattern
Generator (CPG) in the spinal cord, effectively coordinating both frequencies
and amplitudes of the CPG to produce rhythmic output (Rhythm Generation), which
is then mapped to a Pattern Formation (PF) layer. Across different robots, the
only varying component is the PF layer, which adjusts the scaling parameters
for the stride height and length. Subsequently, we evaluate the sim-to-real
transfer by testing the single policy on both the Unitree Go1 and A1 robots.
Remarkably, we observe robust performance, even when adding a 15 kg load,
equivalent to 125% of the A1 robot's nominal mass.
Related papers
- Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation [49.03165169369552]
By training a single policy across many different kinds of robots, a robot learning method can leverage much broader and more diverse datasets.
We propose CrossFormer, a scalable and flexible transformer-based policy that can consume data from any embodiment.
We demonstrate that the same network weights can control vastly different robots, including single and dual arm manipulation systems, wheeled robots, quadcopters, and quadrupeds.
arXiv Detail & Related papers (2024-08-21T17:57:51Z) - Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots.
We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing.
This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
arXiv Detail & Related papers (2024-01-30T10:48:43Z) - Human-Robot Skill Transfer with Enhanced Compliance via Dynamic Movement
Primitives [1.7901837062462316]
We introduce a systematic method to extract the dynamic features from human demonstration to auto-tune the parameters in the Dynamic Movement Primitives framework.
Our method was implemented into an actual human-robot setup to extract human dynamic features and used to regenerate the robot trajectories following both LfD and RL.
arXiv Detail & Related papers (2023-04-12T08:48:28Z) - Universal Morphology Control via Contextual Modulation [52.742056836818136]
Learning a universal policy across different robot morphologies can significantly improve learning efficiency and generalization in continuous control.
Existing methods utilize graph neural networks or transformers to handle heterogeneous state and action spaces across different morphologies.
We propose a hierarchical architecture to better model this dependency via contextual modulation.
arXiv Detail & Related papers (2023-02-22T00:04:12Z) - Low-Rank Modular Reinforcement Learning via Muscle Synergy [25.120547719120765]
Modular Reinforcement Learning (RL) decentralizes the control of multi-joint robots by learning policies for each actuator.
We propose a Synergy-Oriented LeARning (SOLAR) framework that exploits the redundant nature of DoF in robot control.
arXiv Detail & Related papers (2022-10-26T16:01:31Z) - Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement
Learning [18.873152528330063]
We present a reinforcement learning (RL) framework that enables quadrupedal robots to perform soccer goalkeeping tasks in the real world.
Soccer goalkeeping using quadrupeds is a challenging problem, that combines highly dynamic locomotion with precise and fast non-prehensile object (ball) manipulation.
We deploy the proposed framework on a Mini Cheetah quadrupedal robot and demonstrate the effectiveness of our framework for various agile interceptions of a fast-moving ball in the real world.
arXiv Detail & Related papers (2022-10-10T04:54:55Z) - GenLoco: Generalized Locomotion Controllers for Quadrupedal Robots [87.32145104894754]
We introduce a framework for training generalized locomotion (GenLoco) controllers for quadrupedal robots.
Our framework synthesizes general-purpose locomotion controllers that can be deployed on a large variety of quadrupedal robots.
We show that our models acquire more general control strategies that can be directly transferred to novel simulated and real-world robots.
arXiv Detail & Related papers (2022-09-12T15:14:32Z) - MetaMorph: Learning Universal Controllers with Transformers [45.478223199658785]
In robotics we primarily train a single robot for a single task.
modular robot systems now allow for the flexible combination of general-purpose building blocks into task optimized morphologies.
We propose MetaMorph, a Transformer based approach to learn a universal controller over a modular robot design space.
arXiv Detail & Related papers (2022-03-22T17:58:31Z) - REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy
Transfer [57.045140028275036]
We consider the problem of transferring a policy across two different robots with significantly different parameters such as kinematics and morphology.
Existing approaches that train a new policy by matching the action or state transition distribution, including imitation learning methods, fail due to optimal action and/or state distribution being mismatched in different robots.
We propose a novel method named $REvolveR$ of using continuous evolutionary models for robotic policy transfer implemented in a physics simulator.
arXiv Detail & Related papers (2022-02-10T18:50:25Z) - Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic
Platforms [60.59764170868101]
Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform.
We formulate it as a few-shot meta-learning problem where the goal is to find a model that captures the common structure shared across different robotic platforms.
We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots.
arXiv Detail & Related papers (2021-03-05T14:16:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.