Sim2Real for Peg-Hole Insertion with Eye-in-Hand Camera

• URL: http://arxiv.org/abs/2005.14401v1
• Date: Fri, 29 May 2020 05:58:54 GMT
• Title: Sim2Real for Peg-Hole Insertion with Eye-in-Hand Camera
• Authors: Damian Bogunowicz, Aleksandr Rybnikov, Komal Vendidandi and Fedor Chervinskii
• Abstract summary: We use a simulator to learn the peg-hole insertion problem and then transfer the learned model to the real robot. We show that the transferred policy, which only takes RGB-D and joint information (proprioception) can perform well on the real robot.
• Score: 58.720142291102135
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Even though the peg-hole insertion is one of the well-studied problems in robotics, it still remains a challenge for robots, especially when it comes to flexibility and the ability to generalize. Successful completion of the task requires combining several modalities to cope with the complexity of the real world. In our work, we focus on the visual aspect of the problem and employ the strategy of learning an insertion task in a simulator. We use Deep Reinforcement Learning to learn the policy end-to-end and then transfer the learned model to the real robot, without any additional fine-tuning. We show that the transferred policy, which only takes RGB-D and joint information (proprioception) can perform well on the real robot.

Related papers

• $π_0$: A Vision-Language-Action Flow Model for General Robot Control [77.32743739202543]
  We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people, and its ability to acquire new skills via fine-tuning.
  arXiv  Detail & Related papers  (2024-10-31T17:22:30Z)
• VITAL: Visual Teleoperation to Enhance Robot Learning through Human-in-the-Loop Corrections [10.49712834719005]
  We propose a low-cost visual teleoperation system for bimanual manipulation tasks, called VITAL. Our approach leverages affordable hardware and visual processing techniques to collect demonstrations. We enhance the generalizability and robustness of the learned policies by utilizing both real and simulated environments.
  arXiv  Detail & Related papers  (2024-07-30T23:29:47Z)
• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot [56.130215236125224]
  A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception.
  arXiv  Detail & Related papers  (2023-07-02T15:33:31Z)
• Lifelong Robotic Reinforcement Learning by Retaining Experiences [61.79346922421323]
  Many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times. In this work, we study a practical sequential multi-task RL problem motivated by the practical constraints of physical robotic systems. We derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set.
  arXiv  Detail & Related papers  (2021-09-19T18:00:51Z)
• Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spaces [8.692025477306212]
  Reinforcement learning algorithms have proven to be capable of solving complicated robotics tasks in an end-to-end fashion. We propose a framework combining the learning of a low-dimensional state representation, from high-dimensional observations coming from the robot's raw sensory readings, with the learning of the optimal policy.
  arXiv  Detail & Related papers  (2021-07-04T15:42:01Z)
• Meta-Reinforcement Learning for Robotic Industrial Insertion Tasks [70.56451186797436]
  We study how to use meta-reinforcement learning to solve the bulk of the problem in simulation. We demonstrate our approach by training an agent to successfully perform challenging real-world insertion tasks.
  arXiv  Detail & Related papers  (2020-04-29T18:00:22Z)
• The Ingredients of Real-World Robotic Reinforcement Learning [71.92831985295163]
  We discuss the elements that are needed for a robotic learning system that can continually and autonomously improve with data collected in the real world. We propose a particular instantiation of such a system, using dexterous manipulation as our case study. We demonstrate that our complete system can learn without any human intervention, acquiring a variety of vision-based skills with a real-world three-fingered hand.
  arXiv  Detail & Related papers  (2020-04-27T03:36:10Z)
• SQUIRL: Robust and Efficient Learning from Video Demonstration of Long-Horizon Robotic Manipulation Tasks [8.756012472587601]
  Deep reinforcement learning (RL) can be used to learn complex manipulation tasks. RL requires the robot to collect a large amount of real-world experience. S SQUIRL performs a new but related long-horizon task robustly given only a single video demonstration.
  arXiv  Detail & Related papers  (2020-03-10T20:26:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.