Learning Visual Tracking and Reaching with Deep Reinforcement Learning on a UR10e Robotic Arm

• URL: http://arxiv.org/abs/2308.14652v1
• Date: Mon, 28 Aug 2023 15:34:43 GMT
• Title: Learning Visual Tracking and Reaching with Deep Reinforcement Learning on a UR10e Robotic Arm
• Authors: Colin Bellinger, Laurence Lamarche-Cliche
• Abstract summary: Reinforcement learning algorithms provide the potential to enable robots to learn optimal solutions to complete new tasks without reprogramming them. Current state-of-the-art in reinforcement learning relies on fast simulations and parallelization to achieve optimal performance. This report outlines our initial research into the application of deep reinforcement learning on an industrial UR10e robot.
• Score: 2.2168889407389445
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As technology progresses, industrial and scientific robots are increasingly being used in diverse settings. In many cases, however, programming the robot to perform such tasks is technically complex and costly. To maximize the utility of robots in industrial and scientific settings, they require the ability to quickly shift from one task to another. Reinforcement learning algorithms provide the potential to enable robots to learn optimal solutions to complete new tasks without directly reprogramming them. The current state-of-the-art in reinforcement learning, however, generally relies on fast simulations and parallelization to achieve optimal performance. These are often not possible in robotics applications. Thus, a significant amount of research is required to facilitate the efficient and safe, training and deployment of industrial and scientific reinforcement learning robots. This technical report outlines our initial research into the application of deep reinforcement learning on an industrial UR10e robot. The report describes the reinforcement learning environments created to facilitate policy learning with the UR10e, a robotic arm from Universal Robots, and presents our initial results in training deep Q-learning and proximal policy optimization agents on the developed reinforcement learning environments. Our results show that proximal policy optimization learns a better, more stable policy with less data than deep Q-learning. The corresponding code for this work is available at \url{https://github.com/cbellinger27/bendRL_reacher_tracker}

Related papers

• SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning [85.21378553454672]
  We develop a library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment. We find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation. These policies achieve perfect or near-perfect success rates, extreme robustness even under perturbations, and exhibit emergent robustness recovery and correction behaviors.
  arXiv  Detail & Related papers  (2024-01-29T10:01:10Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)
• Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment [25.186525630548356]
  Sirius is a principled framework for humans and robots to collaborate through a division of work. Partially autonomous robots are tasked with handling a major portion of decision-making where they work reliably. We introduce a new learning algorithm to improve the policy's performance on the data collected from the task executions.
  arXiv  Detail & Related papers  (2022-11-15T18:53:39Z)
• Accelerating Robot Learning of Contact-Rich Manipulations: A Curriculum Learning Study [4.045850174820418]
  This paper presents a study for accelerating robot learning of contact-rich manipulation tasks based on Curriculum Learning combined with Domain Randomization (DR) We tackle complex industrial assembly tasks with position-controlled robots, such as insertion tasks. Results also show that even when training only in simulation with toy tasks, our method can learn policies that can be transferred to the real-world robot.
  arXiv  Detail & Related papers  (2022-04-27T11:08:39Z)
• Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives [92.0321404272942]
  Reinforcement learning can be used to build general-purpose robotic systems. However, training RL agents to solve robotics tasks still remains challenging. In this work, we manually specify a library of robot action primitives (RAPS), parameterized with arguments that are learned by an RL policy. We find that our simple change to the action interface substantially improves both the learning efficiency and task performance.
  arXiv  Detail & Related papers  (2021-10-28T17:59:30Z)
• Lifelong Robotic Reinforcement Learning by Retaining Experiences [61.79346922421323]
  Many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times. In this work, we study a practical sequential multi-task RL problem motivated by the practical constraints of physical robotic systems. We derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set.
  arXiv  Detail & Related papers  (2021-09-19T18:00:51Z)
• A Framework for Efficient Robotic Manipulation [79.10407063260473]
  We show that a single robotic arm can learn sparse-reward manipulation policies from pixels. We show that, given only 10 demonstrations, a single robotic arm can learn sparse-reward manipulation policies from pixels.
  arXiv  Detail & Related papers  (2020-12-14T22:18:39Z)
• SQUIRL: Robust and Efficient Learning from Video Demonstration of Long-Horizon Robotic Manipulation Tasks [8.756012472587601]
  Deep reinforcement learning (RL) can be used to learn complex manipulation tasks. RL requires the robot to collect a large amount of real-world experience. S SQUIRL performs a new but related long-horizon task robustly given only a single video demonstration.
  arXiv  Detail & Related papers  (2020-03-10T20:26:26Z)
• Efficient reinforcement learning control for continuum robots based on Inexplicit Prior Knowledge [3.3645162441357437]
  We propose an efficient reinforcement learning method based on inexplicit prior knowledge. By using our method, we can achieve active visual tracking and distance maintenance of a tendon-driven robot.
  arXiv  Detail & Related papers  (2020-02-26T15:47:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.