Learning Bipedal Walking for Humanoids with Current Feedback
- URL: http://arxiv.org/abs/2303.03724v2
- Date: Mon, 7 Aug 2023 05:52:36 GMT
- Title: Learning Bipedal Walking for Humanoids with Current Feedback
- Authors: Rohan Pratap Singh, Zhaoming Xie, Pierre Gergondet, Fumio Kanehiro
- Abstract summary: We present an approach for overcoming the sim2real gap issue for humanoid robots arising from inaccurate torque-tracking at the actuator level.
Our approach successfully trains a unified, end-to-end policy in simulation that can be deployed on a real HRP-5P humanoid robot to achieve bipedal locomotion.
- Score: 5.429166905724048
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recent advances in deep reinforcement learning (RL) based techniques combined
with training in simulation have offered a new approach to developing robust
controllers for legged robots. However, the application of such approaches to
real hardware has largely been limited to quadrupedal robots with direct-drive
actuators and light-weight bipedal robots with low gear-ratio transmission
systems. Application to real, life-sized humanoid robots has been less common
arguably due to a large sim2real gap. In this paper, we present an approach for
effectively overcoming the sim2real gap issue for humanoid robots arising from
inaccurate torque-tracking at the actuator level. Our key idea is to utilize
the current feedback from the actuators on the real robot, after training the
policy in a simulation environment artificially degraded with poor
torque-tracking. Our approach successfully trains a unified, end-to-end policy
in simulation that can be deployed on a real HRP-5P humanoid robot to achieve
bipedal locomotion. Through ablations, we also show that a feedforward policy
architecture combined with targeted dynamics randomization is sufficient for
zero-shot sim2real success, thus eliminating the need for computationally
expensive, memory-based network architectures. Finally, we validate the
robustness of the proposed RL policy by comparing its performance against a
conventional model-based controller for walking on uneven terrain with the real
robot.
Related papers
- Learning to navigate efficiently and precisely in real environments [14.52507964172957]
Embodied AI literature focuses on end-to-end agents trained in simulators like Habitat or AI-Thor.
In this work we explore end-to-end training of agents in simulation in settings which minimize the sim2real gap.
arXiv Detail & Related papers (2024-01-25T17:50:05Z) - Towards Transferring Tactile-based Continuous Force Control Policies
from Simulation to Robot [19.789369416528604]
grasp force control aims to manipulate objects safely by limiting the amount of force exerted on the object.
Prior works have either hand-modeled their force controllers, employed model-based approaches, or have not shown sim-to-real transfer.
We propose a model-free deep reinforcement learning approach trained in simulation and then transferred to the robot without further fine-tuning.
arXiv Detail & Related papers (2023-11-13T11:29:06Z) - Human-Robot Skill Transfer with Enhanced Compliance via Dynamic Movement
Primitives [1.7901837062462316]
We introduce a systematic method to extract the dynamic features from human demonstration to auto-tune the parameters in the Dynamic Movement Primitives framework.
Our method was implemented into an actual human-robot setup to extract human dynamic features and used to regenerate the robot trajectories following both LfD and RL.
arXiv Detail & Related papers (2023-04-12T08:48:28Z) - Learning Human-to-Robot Handovers from Point Clouds [63.18127198174958]
We propose the first framework to learn control policies for vision-based human-to-robot handovers.
We show significant performance gains over baselines on a simulation benchmark, sim-to-sim transfer and sim-to-real transfer.
arXiv Detail & Related papers (2023-03-30T17:58:36Z) - Obstacle Avoidance for Robotic Manipulator in Joint Space via Improved
Proximal Policy Optimization [6.067589886362815]
In this paper, we train a deep neural network via an improved Proximal Policy Optimization (PPO) algorithm to map from task space to joint space for a 6-DoF manipulator.
Since training such a task in real-robot is time-consuming and strenuous, we develop a simulation environment to train the model.
Experimental results showed that using our method, the robot was capable of tracking a single target or reaching multiple targets in unstructured environments.
arXiv Detail & Related papers (2022-10-03T10:21:57Z) - Active Predicting Coding: Brain-Inspired Reinforcement Learning for
Sparse Reward Robotic Control Problems [79.07468367923619]
We propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC)
We design an agent built completely from powerful predictive coding/processing circuits that facilitate dynamic, online learning from sparse rewards.
We show that our proposed ActPC agent performs well in the face of sparse (extrinsic) reward signals and is competitive with or outperforms several powerful backprop-based RL approaches.
arXiv Detail & Related papers (2022-09-19T16:49:32Z) - GenLoco: Generalized Locomotion Controllers for Quadrupedal Robots [87.32145104894754]
We introduce a framework for training generalized locomotion (GenLoco) controllers for quadrupedal robots.
Our framework synthesizes general-purpose locomotion controllers that can be deployed on a large variety of quadrupedal robots.
We show that our models acquire more general control strategies that can be directly transferred to novel simulated and real-world robots.
arXiv Detail & Related papers (2022-09-12T15:14:32Z) - REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy
Transfer [57.045140028275036]
We consider the problem of transferring a policy across two different robots with significantly different parameters such as kinematics and morphology.
Existing approaches that train a new policy by matching the action or state transition distribution, including imitation learning methods, fail due to optimal action and/or state distribution being mismatched in different robots.
We propose a novel method named $REvolveR$ of using continuous evolutionary models for robotic policy transfer implemented in a physics simulator.
arXiv Detail & Related papers (2022-02-10T18:50:25Z) - Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform.
We produce a closed-loop controller to reactively push objects in a continuous action space.
We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
arXiv Detail & Related papers (2021-11-15T18:50:04Z) - Robot Learning from Randomized Simulations: A Review [59.992761565399185]
Deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data.
State-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive.
We focus on a technique named 'domain randomization' which is a method for learning from randomized simulations.
arXiv Detail & Related papers (2021-11-01T13:55:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.