Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

• URL: http://arxiv.org/abs/2304.13705v1
• Date: Sun, 23 Apr 2023 19:10:53 GMT
• Title: Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
• Authors: Tony Z. Zhao, Vikash Kumar, Sergey Levine, Chelsea Finn
• Abstract summary: Fine manipulation tasks, such as threading cable ties or slotting a battery, are notoriously difficult for robots. We present a low-cost system that performs end-to-end imitation learning directly from real demonstrations. We develop a simple yet novel algorithm, Action Chunking with Transformers, which learns a generative model over action sequences.
• Score: 132.39281056124312
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fine manipulation tasks, such as threading cable ties or slotting a battery, are notoriously difficult for robots because they require precision, careful coordination of contact forces, and closed-loop visual feedback. Performing these tasks typically requires high-end robots, accurate sensors, or careful calibration, which can be expensive and difficult to set up. Can learning enable low-cost and imprecise hardware to perform these fine manipulation tasks? We present a low-cost system that performs end-to-end imitation learning directly from real demonstrations, collected with a custom teleoperation interface. Imitation learning, however, presents its own challenges, particularly in high-precision domains: errors in the policy can compound over time, and human demonstrations can be non-stationary. To address these challenges, we develop a simple yet novel algorithm, Action Chunking with Transformers (ACT), which learns a generative model over action sequences. ACT allows the robot to learn 6 difficult tasks in the real world, such as opening a translucent condiment cup and slotting a battery with 80-90% success, with only 10 minutes worth of demonstrations. Project website: https://tonyzhaozh.github.io/aloha/

Related papers

• Zero-Cost Whole-Body Teleoperation for Mobile Manipulation [8.71539730969424]
  MoMa-Teleop is a novel teleoperation method that delegates the base motions to a reinforcement learning agent. We demonstrate that our approach results in a significant reduction in task completion time across a variety of robots and tasks.
  arXiv  Detail & Related papers  (2024-09-23T15:09:45Z)
• VITAL: Visual Teleoperation to Enhance Robot Learning through Human-in-the-Loop Corrections [10.49712834719005]
  We propose a low-cost visual teleoperation system for bimanual manipulation tasks, called VITAL. Our approach leverages affordable hardware and visual processing techniques to collect demonstrations. We enhance the generalizability and robustness of the learned policies by utilizing both real and simulated environments.
  arXiv  Detail & Related papers  (2024-07-30T23:29:47Z)
• Offline Imitation Learning Through Graph Search and Retrieval [57.57306578140857]
  Imitation learning is a powerful machine learning algorithm for a robot to acquire manipulation skills. We propose GSR, a simple yet effective algorithm that learns from suboptimal demonstrations through Graph Search and Retrieval. GSR can achieve a 10% to 30% higher success rate and over 30% higher proficiency compared to baselines.
  arXiv  Detail & Related papers  (2024-07-22T06:12:21Z)
• Learning Variable Compliance Control From a Few Demonstrations for Bimanual Robot with Haptic Feedback Teleoperation System [5.497832119577795]
  dexterous, contact-rich manipulation tasks using rigid robots is a significant challenge in robotics. Compliance control schemes have been introduced to mitigate these issues by controlling forces via external sensors. Learning from Demonstrations offers an intuitive alternative, allowing robots to learn manipulations through observed actions.
  arXiv  Detail & Related papers  (2024-06-21T09:03:37Z)
• Harmonic Mobile Manipulation [35.82197562695662]
  HarmonicMM is an end-to-end learning method that optimize both navigation and manipulation. Our contributions include a new benchmark for mobile manipulation and the successful deployment with only RGB visual observation.
  arXiv  Detail & Related papers  (2023-12-11T18:54:42Z)
• Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration [68.94506047556412]
  We propose to leverage a sequential bias to learn control policies for complex robotic tasks using a single demonstration. We show that DCIL-II can solve with unprecedented sample efficiency some challenging simulated tasks such as humanoid locomotion and stand-up.
  arXiv  Detail & Related papers  (2022-11-09T10:28:40Z)
• Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation [13.135013586592585]
  'Dexterous Made Easy' (DIME) is a new imitation learning framework for dexterous manipulation. DIME only requires a single RGB camera to observe a human operator and teleoperate our robotic hand. On both simulation and real robot benchmarks we demonstrate that DIME can be used to solve complex, in-hand manipulation tasks.
  arXiv  Detail & Related papers  (2022-03-24T17:58:54Z)
• Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation [55.31301153979621]
  We tackle real-world long-horizon robot manipulation tasks through skill discovery. We present a bottom-up approach to learning a library of reusable skills from unsegmented demonstrations. Our method has shown superior performance over state-of-the-art imitation learning methods in multi-stage manipulation tasks.
  arXiv  Detail & Related papers  (2021-09-28T16:18:54Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)
• Sim2Real for Peg-Hole Insertion with Eye-in-Hand Camera [58.720142291102135]
  We use a simulator to learn the peg-hole insertion problem and then transfer the learned model to the real robot. We show that the transferred policy, which only takes RGB-D and joint information (proprioception) can perform well on the real robot.
  arXiv  Detail & Related papers  (2020-05-29T05:58:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.