Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasets

• URL: http://arxiv.org/abs/2310.04517v1
• Date: Fri, 6 Oct 2023 18:26:09 GMT
• Title: Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasets
• Authors: Johann Huber, Fran\c{c}ois H\'el\'enon, Hippolyte Watrelot, Faiz Ben Amar and St\'ephane Doncieux
• Abstract summary: The present paper investigates how automatically generated grasps can be exploited in the real world. More than 7000 reach-and-grasp trajectories have been generated with Quality-Diversity (QD) methods on 3 different arms and grippers, including parallel fingers and a dexterous hand, and tested in the real world. A QD approach has finally been proposed for making grasps more robust to domain randomization, resulting in a transfer ratio of 84% on the Franka Research 3 arm.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic grasping refers to making a robotic system pick an object by applying forces and torques on its surface. Many recent studies use data-driven approaches to address grasping, but the sparse reward nature of this task made the learning process challenging to bootstrap. To avoid constraining the operational space, an increasing number of works propose grasping datasets to learn from. But most of them are limited to simulations. The present paper investigates how automatically generated grasps can be exploited in the real world. More than 7000 reach-and-grasp trajectories have been generated with Quality-Diversity (QD) methods on 3 different arms and grippers, including parallel fingers and a dexterous hand, and tested in the real world. Conducted analysis on the collected measure shows correlations between several Domain Randomization-based quality criteria and sim-to-real transferability. Key challenges regarding the reality gap for grasping have been identified, stressing matters on which researchers on grasping should focus in the future. A QD approach has finally been proposed for making grasps more robust to domain randomization, resulting in a transfer ratio of 84% on the Franka Research 3 arm.

Related papers

• Single-Shot Learning of Stable Dynamical Systems for Long-Horizon Manipulation Tasks [48.54757719504994]
  This paper focuses on improving task success rates while reducing the amount of training data needed. Our approach introduces a novel method that segments long-horizon demonstrations into discrete steps defined by waypoints and subgoals. We validate our approach through both simulation and real-world experiments, demonstrating effective transfer from simulation to physical robotic platforms.
  arXiv  Detail & Related papers  (2024-10-01T19:49:56Z)
• Speeding up 6-DoF Grasp Sampling with Quality-Diversity [1.533848041901807]
  Quality-Diversity (QD) algorithms optimize a set of solutions to get diverse, high-performing solutions to a given problem. Experiments conducted on 4 grippers with 2-to-5 fingers on standard objects show that QD outperforms commonly used methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-10T10:58:54Z)
• Latent Task-Specific Graph Network Simulators [16.881339139068018]
  Graph Network Simulators (GNSs) pose an efficient alternative to traditional physics-based simulators. We frame mesh-based simulation as a meta-learning problem and use a recent Bayesian meta-learning method to improve GNSs adaptability to new scenarios. We validate the effectiveness of our approach through various experiments, performing on par with or better than established baseline methods.
  arXiv  Detail & Related papers  (2023-11-09T10:30:51Z)
• PU-Ray: Domain-Independent Point Cloud Upsampling via Ray Marching on Neural Implicit Surface [5.78575346449322]
  We propose a new ray-based upsampling approach with an arbitrary rate, where a depth prediction is made for each query ray and its corresponding patch. Our novel method simulates the sphere-tracing ray marching algorithm on the neural implicit surface defined with an unsigned distance function (UDF) The rule-based mid-point query sampling method generates more evenly distributed points without requiring an end-to-end model trained using a nearest-neighbor-based reconstruction loss function.
  arXiv  Detail & Related papers  (2023-10-12T22:45:03Z)
• Robust Visual Sim-to-Real Transfer for Robotic Manipulation [79.66851068682779]
  Learning visuomotor policies in simulation is much safer and cheaper than in the real world. However, due to discrepancies between the simulated and real data, simulator-trained policies often fail when transferred to real robots. One common approach to bridge the visual sim-to-real domain gap is domain randomization (DR)
  arXiv  Detail & Related papers  (2023-07-28T05:47:24Z)
• Towards Precise Model-free Robotic Grasping with Sim-to-Real Transfer Learning [11.470950882435927]
  We present an end-to-end robotic grasping network with a grasp. In physical robotic experiments, our grasping framework grasped single known objects and novel complex-shaped household objects with a success rate of 90.91%. The proposed grasping framework outperformed two state-of-the-art methods in both known and unknown object robotic grasping.
  arXiv  Detail & Related papers  (2023-01-28T16:57:19Z)
• Understanding Domain Randomization for Sim-to-real Transfer [41.33483293243257]
  We propose a theoretical framework for sim-to-real transfers, in which the simulator is modeled as a set of MDPs with tunable parameters. We prove that sim-to-real transfer can succeed under mild conditions without any real-world training samples.
  arXiv  Detail & Related papers  (2021-10-07T07:45:59Z)
• TraND: Transferable Neighborhood Discovery for Unsupervised Cross-domain Gait Recognition [77.77786072373942]
  This paper proposes a Transferable Neighborhood Discovery (TraND) framework to bridge the domain gap for unsupervised cross-domain gait recognition. We design an end-to-end trainable approach to automatically discover the confident neighborhoods of unlabeled samples in the latent space. Our method achieves state-of-the-art results on two public datasets, i.e., CASIA-B and OU-LP.
  arXiv  Detail & Related papers  (2021-02-09T03:07:07Z)
• Intervention Design for Effective Sim2Real Transfer [48.9711031777803]
  This work addresses the recent success of domain randomization and data augmentation for the sim2real setting. We explain this success through the lens of causal inference, positioning domain randomization and data augmentation as interventions on the environment.
  arXiv  Detail & Related papers  (2020-12-03T16:38:54Z)
• SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
  We propose advances that address two key challenges in future trajectory prediction. multimodality in both training data and predictions and constant time inference regardless of number of agents.
  arXiv  Detail & Related papers  (2020-07-26T08:17:10Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.