Constrained Reinforcement Learning for Dexterous Manipulation

• URL: http://arxiv.org/abs/2301.09766v1
• Date: Tue, 24 Jan 2023 00:31:28 GMT
• Title: Constrained Reinforcement Learning for Dexterous Manipulation
• Authors: Abhineet Jain, Jack Kolb and Harish Ravichandar
• Abstract summary: We investigate the effects of adding position-based constraints to a 24-DOF robot hand learning to perform object relocation. We find that a simple geometric constraint can ensure the robot learns to move towards the object sooner than without constraints. These findings shed light on how simple constraints can help robots achieve sensible and safe behavior quickly and ease concerns surrounding hardware deployment.
• Score: 0.6193838300896449
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing learning approaches to dexterous manipulation use demonstrations or interactions with the environment to train black-box neural networks that provide little control over how the robot learns the skills or how it would perform post training. These approaches pose significant challenges when implemented on physical platforms given that, during initial stages of training, the robot's behavior could be erratic and potentially harmful to its own hardware, the environment, or any humans in the vicinity. A potential way to address these limitations is to add constraints during learning that restrict and guide the robot's behavior during training as well as roll outs. Inspired by the success of constrained approaches in other domains, we investigate the effects of adding position-based constraints to a 24-DOF robot hand learning to perform object relocation using Constrained Policy Optimization. We find that a simple geometric constraint can ensure the robot learns to move towards the object sooner than without constraints. Further, training with this constraint requires a similar number of samples as its unconstrained counterpart to master the skill. These findings shed light on how simple constraints can help robots achieve sensible and safe behavior quickly and ease concerns surrounding hardware deployment. We also investigate the effects of the strictness of these constraints and report findings that provide insights into how different degrees of strictness affect learning outcomes. Our code is available at https://github.com/GT-STAR-Lab/constrained-rl-dexterous-manipulation.

Related papers

• Safe Reinforcement Learning on the Constraint Manifold: Theory and Applications [21.98309272057848]
  We show how we can impose complex safety constraints on learning-based robotics systems in a principled manner. Our approach is based on the concept of the Constraint Manifold, representing the set of safe robot configurations. We demonstrate the method's effectiveness in a real-world Robot Air Hockey task.
  arXiv  Detail & Related papers  (2024-04-13T20:55:15Z)
• Learning Shared Safety Constraints from Multi-task Demonstrations [53.116648461888936]
  We show how to learn constraints from expert demonstrations of safe task completion. We learn constraints that forbid highly rewarding behavior that the expert could have taken but chose not to. We validate our method with simulation experiments on high-dimensional continuous control tasks.
  arXiv  Detail & Related papers  (2023-09-01T19:37:36Z)
• Learning Vision-based Pursuit-Evasion Robot Policies [54.52536214251999]
  We develop a fully-observable robot policy that generates supervision for a partially-observable one. We deploy our policy on a physical quadruped robot with an RGB-D camera on pursuit-evasion interactions in the wild.
  arXiv  Detail & Related papers  (2023-08-30T17:59:05Z)
• Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Exploration [8.343657309038285]
  Reinforcement Learning is a powerful framework for developing such robot controllers. We propose a multimodal exploration approach through categorical distributions, which enables us to train planar pushing RL policies. We show that the learned policies are robust to external disturbances and observation noise, and scale to tasks with multiple pushers.
  arXiv  Detail & Related papers  (2023-08-04T16:55:00Z)
• DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training [10.808149303943948]
  We learn dexterous object manipulation using simulated one- or two-armed robots equipped with multi-fingered hand end-effectors. We introduce a decentralized Population-Based Training (PBT) algorithm that allows us to massively amplify the exploration capabilities of deep reinforcement learning.
  arXiv  Detail & Related papers  (2023-05-20T07:25:27Z)
• Learning and Adapting Agile Locomotion Skills by Transferring Experience [71.8926510772552]
  We propose a framework for training complex robotic skills by transferring experience from existing controllers to jumpstart learning new tasks. We show that our method enables learning complex agile jumping behaviors, navigating to goal locations while walking on hind legs, and adapting to new environments.
  arXiv  Detail & Related papers  (2023-04-19T17:37:54Z)
• Differentiable Constrained Imitation Learning for Robot Motion Planning and Control [0.26999000177990923]
  We develop a framework for constraint robotic motion planning and control, as well as traffic agent simulation. We focus on mobile robot and automated driving applications. Simulated experiments of mobile robot navigation and automated driving provide evidence for the performance of the proposed method.
  arXiv  Detail & Related papers  (2022-10-21T08:19:45Z)
• Vision-Based Mobile Robotics Obstacle Avoidance With Deep Reinforcement Learning [49.04274612323564]
  Obstacle avoidance is a fundamental and challenging problem for autonomous navigation of mobile robots. In this paper, we consider the problem of obstacle avoidance in simple 3D environments where the robot has to solely rely on a single monocular camera. We tackle the obstacle avoidance problem as a data-driven end-to-end deep learning approach.
  arXiv  Detail & Related papers  (2021-03-08T13:05:46Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)
• DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics [44.62475518267084]
  We present a developmental cognitive architecture to bootstrap this redescription process stage by stage, build new state representations with appropriate motivations, and transfer the acquired knowledge across domains or tasks or even across robots.
  arXiv  Detail & Related papers  (2020-05-13T09:29:40Z)
• Meta-Reinforcement Learning for Robotic Industrial Insertion Tasks [70.56451186797436]
  We study how to use meta-reinforcement learning to solve the bulk of the problem in simulation. We demonstrate our approach by training an agent to successfully perform challenging real-world insertion tasks.
  arXiv  Detail & Related papers  (2020-04-29T18:00:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.