Synthetic Data Generation for Bridging Sim2Real Gap in a Production Environment

• URL: http://arxiv.org/abs/2311.11039v2
• Date: Fri, 10 May 2024 11:48:33 GMT
• Title: Synthetic Data Generation for Bridging Sim2Real Gap in a Production Environment
• Authors: Parth Rawal, Mrunal Sompura, Wolfgang Hintze,
• Abstract summary: Domain knowledge is vital in bridging the simulation to reality gap in computer vision applications. This paper focuses on synthetic data generation procedures for parts and assemblies used in a production environment.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthetic data is being used lately for training deep neural networks in computer vision applications such as object detection, object segmentation and 6D object pose estimation. Domain randomization hereby plays an important role in reducing the simulation to reality gap. However, this generalization might not be effective in specialized domains like a production environment involving complex assemblies. Either the individual parts, trained with synthetic images, are integrated in much larger assemblies making them indistinguishable from their counterparts and result in false positives or are partially occluded just enough to give rise to false negatives. Domain knowledge is vital in these cases and if conceived effectively while generating synthetic data, can show a considerable improvement in bridging the simulation to reality gap. This paper focuses on synthetic data generation procedures for parts and assemblies used in a production environment. The basic procedures for synthetic data generation and their various combinations are evaluated and compared on images captured in a production environment, where results show up to 15% improvement using combinations of basic procedures. Reducing the simulation to reality gap in this way can aid to utilize the true potential of robot assisted production using artificial intelligence.

Related papers

• WasteGAN: Data Augmentation for Robotic Waste Sorting through Generative Adversarial Networks [7.775894876221921]
  We introduce a data augmentation method based on a novel GAN architecture called wasteGAN. The proposed method allows to increase the performance of semantic segmentation models, starting from a very limited bunch of labeled examples. We then leverage the higher-quality segmentation masks predicted from models trained on the wasteGAN synthetic data to compute semantic-aware grasp poses.
  arXiv  Detail & Related papers  (2024-09-25T15:04:21Z)
• Efficient Data Collection for Robotic Manipulation via Compositional Generalization [70.76782930312746]
  We show that policies can compose environmental factors from their data to succeed when encountering unseen factor combinations. We propose better in-domain data collection strategies that exploit composition. We provide videos at http://iliad.stanford.edu/robot-data-comp/.
  arXiv  Detail & Related papers  (2024-03-08T07:15:38Z)
• Synthetic data, real errors: how (not) to publish and use synthetic data [86.65594304109567]
  We show how the generative process affects the downstream ML task. We introduce Deep Generative Ensemble (DGE) to approximate the posterior distribution over the generative process model parameters.
  arXiv  Detail & Related papers  (2023-05-16T07:30:29Z)
• ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning [102.46382882098847]
  We first investigate the effects of synthetic data in synthetic-to-real novel view synthesis. We propose to introduce geometry-aware contrastive learning to learn multi-view consistent features with geometric constraints. Our method can render images with higher quality and better fine-grained details, outperforming existing generalizable novel view synthesis methods in terms of PSNR, SSIM, and LPIPS.
  arXiv  Detail & Related papers  (2023-03-20T12:06:14Z)
• Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances [76.34037366117234]
  We introduce a new dataset called Robot Control Gestures (RoCoG-v2) The dataset is composed of both real and synthetic videos from seven gesture classes. We present results using state-of-the-art action recognition and domain adaptation algorithms.
  arXiv  Detail & Related papers  (2023-03-17T23:23:55Z)
• A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptation [42.2398858786125]
  Deep learning in computer vision has achieved great success with the price of large-scale labeled training data. The uncontrollable data collection process produces non-IID training and test data, where undesired duplication may exist. To circumvent them, an alternative is to generate synthetic data via 3D rendering with domain randomization.
  arXiv  Detail & Related papers  (2023-03-16T09:03:52Z)
• TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments [84.6017003787244]
  This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
  arXiv  Detail & Related papers  (2022-08-16T20:46:08Z)
• Synthetic Dataset Generation for Adversarial Machine Learning Research [0.0]
  Existing adversarial example research focuses on digitally inserted perturbations on top of existing natural image datasets. This construction of adversarial examples is not realistic because it may be difficult, or even impossible, for an attacker to deploy such an attack in the real-world due to sensing and environmental effects. To better understand adversarial examples against cyber-physical systems, we propose approximating the real-world through simulation.
  arXiv  Detail & Related papers  (2022-07-21T19:14:44Z)
• Delving into High-Quality Synthetic Face Occlusion Segmentation Datasets [83.749895930242]
  We propose two techniques for producing high-quality naturalistic synthetic occluded faces. We empirically show the effectiveness and robustness of both methods, even for unseen occlusions. We present two high-resolution real-world occluded face datasets with fine-grained annotations, RealOcc and RealOcc-Wild.
  arXiv  Detail & Related papers  (2022-05-12T17:03:57Z)
• Fake It Till You Make It: Face analysis in the wild using synthetic data alone [9.081019005437309]
  We show that it is possible to perform face-related computer vision in the wild using synthetic data alone. We describe how to combine a procedurally-generated 3D face model with a comprehensive library of hand-crafted assets to render training images with unprecedented realism.
  arXiv  Detail & Related papers  (2021-09-30T13:07:04Z)
• Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming [3.4788711710826083]
  We propose an alternative solution with respect to the common data augmentation methods, applying it to the problem of crop/weed segmentation in precision farming. We create semi-artificial samples by replacing the most relevant object classes (i.e., crop and weeds) with their synthesized counterparts. In addition to RGB data, we take into account also near-infrared (NIR) information, generating four channel multi-spectral synthetic images.
  arXiv  Detail & Related papers  (2020-09-12T08:49:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.