An Efficient Image-to-Image Translation HourGlass-based Architecture for Object Pushing Policy Learning

• URL: http://arxiv.org/abs/2108.01034v1
• Date: Mon, 2 Aug 2021 16:46:08 GMT
• Title: An Efficient Image-to-Image Translation HourGlass-based Architecture for Object Pushing Policy Learning
• Authors: Marco Ewerton, Angel Mart\'inez-Gonz\'alez, Jean-Marc Odobez
• Abstract summary: Humans effortlessly solve pushing tasks in everyday life but unlocking these capabilities remains a challenge in robotics. We present an architecture combining a predictor of which pushes lead to changes in the environment with a state-action value predictor dedicated to the pushing task. We demonstrate in simulation experiments with a UR5 robot arm that our overall architecture helps the DQN learn faster and achieve higher performance.
• Score: 20.77172985076276
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans effortlessly solve pushing tasks in everyday life but unlocking these capabilities remains a challenge in robotics because physics models of these tasks are often inaccurate or unattainable. State-of-the-art data-driven approaches learn to compensate for these inaccuracies or replace the approximated physics models altogether. Nevertheless, approaches like Deep Q-Networks (DQNs) suffer from local optima in large state-action spaces. Furthermore, they rely on well-chosen deep learning architectures and learning paradigms. In this paper, we propose to frame the learning of pushing policies (where to push and how) by DQNs as an image-to-image translation problem and exploit an Hourglass-based architecture. We present an architecture combining a predictor of which pushes lead to changes in the environment with a state-action value predictor dedicated to the pushing task. Moreover, we investigate positional information encoding to learn position-dependent policy behaviors. We demonstrate in simulation experiments with a UR5 robot arm that our overall architecture helps the DQN learn faster and achieve higher performance in a pushing task involving objects with unknown dynamics.

Related papers

• Graphical Object-Centric Actor-Critic [55.2480439325792]
  We propose a novel object-centric reinforcement learning algorithm combining actor-critic and model-based approaches. We use a transformer encoder to extract object representations and graph neural networks to approximate the dynamics of an environment. Our algorithm performs better in a visually complex 3D robotic environment and a 2D environment with compositional structure than the state-of-the-art model-free actor-critic algorithm.
  arXiv  Detail & Related papers  (2023-10-26T06:05:12Z)
• DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools [96.38972082580294]
  DiffSkill is a novel framework that uses a differentiable physics simulator for skill abstraction to solve deformable object manipulation tasks. In particular, we first obtain short-horizon skills using individual tools from a gradient-based simulator. We then learn a neural skill abstractor from the demonstration trajectories which takes RGBD images as input.
  arXiv  Detail & Related papers  (2022-03-31T17:59:38Z)
• Neural Architecture Search for Dense Prediction Tasks in Computer Vision [74.9839082859151]
  Deep learning has led to a rising demand for neural network architecture engineering. neural architecture search (NAS) aims at automatically designing neural network architectures in a data-driven manner rather than manually. NAS has become applicable to a much wider range of problems in computer vision.
  arXiv  Detail & Related papers  (2022-02-15T08:06:50Z)
• Combining Commonsense Reasoning and Knowledge Acquisition to Guide Deep Learning in Robotics [8.566457170664926]
  The architecture described in this paper draws inspiration from research in cognitive systems. Deep network models are being used for many pattern recognition and decision-making tasks in robotics and AI. Our architecture improves reliability of decision making and reduces the effort involved in training data-driven deep network models.
  arXiv  Detail & Related papers  (2022-01-25T12:24:22Z)
• Improving the sample-efficiency of neural architecture search with reinforcement learning [0.0]
  In this work, we would like to contribute to the area of Automated Machine Learning (AutoML) Our focus is on one of the most promising research directions, reinforcement learning. The validation accuracies of the child networks serve as a reward signal for training the controller. We propose to modify this to a more modern and complex algorithm, PPO, which has demonstrated to be faster and more stable in other environments.
  arXiv  Detail & Related papers  (2021-10-13T14:30:09Z)
• Hierarchical Neural Dynamic Policies [50.969565411919376]
  We tackle the problem of generalization to unseen configurations for dynamic tasks in the real world while learning from high-dimensional image input. We use hierarchical deep policy learning framework called Hierarchical Neural Dynamical Policies (H-NDPs) H-NDPs form a curriculum by learning local dynamical system-based policies on small regions in state-space. We show that H-NDPs are easily integrated with both imitation as well as reinforcement learning setups and achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2021-07-12T17:59:58Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)
• Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
  We present a deep imitation learning framework for robotic bimanual manipulation. A core challenge is to generalize the manipulation skills to objects in different locations. We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
  arXiv  Detail & Related papers  (2020-10-11T01:40:03Z)
• 3D_DEN: Open-ended 3D Object Recognition using Dynamically Expandable Networks [0.0]
  We propose a new deep transfer learning approach based on a dynamic architectural method to make robots capable of open-ended learning about new 3D object categories. Experimental results showed that the proposed model outperformed state-of-the-art approaches with regards to accuracy and also substantially minimizes computational overhead.
  arXiv  Detail & Related papers  (2020-09-15T16:44:18Z)
• Robotic Grasp Manipulation Using Evolutionary Computing and Deep Reinforcement Learning [0.0]
  Humans almost immediately know how to manipulate objects for grasping due to learning over the years. In this paper we have taken up the challenge of developing learning based pose estimation by decomposing the problem into both position and orientation learning. Based on our proposed architectures and algorithms, the robot is capable of grasping all rigid body objects having regular shapes.
  arXiv  Detail & Related papers  (2020-01-15T17:23:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.