HaDR: Applying Domain Randomization for Generating Synthetic Multimodal Dataset for Hand Instance Segmentation in Cluttered Industrial Environments

• URL: http://arxiv.org/abs/2304.05826v1
• Date: Wed, 12 Apr 2023 13:02:08 GMT
• Title: HaDR: Applying Domain Randomization for Generating Synthetic Multimodal Dataset for Hand Instance Segmentation in Cluttered Industrial Environments
• Authors: Stefan Grushko, Ale\v{s} Vysock\'y, Jakub Chlebek, Petr Prokop
• Abstract summary: This study uses domain randomization to generate a synthetic RGB-D dataset for training multimodal instance segmentation models. We show that our approach enables the models to outperform corresponding models trained on existing state-of-the-art datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This study uses domain randomization to generate a synthetic RGB-D dataset for training multimodal instance segmentation models, aiming to achieve colour-agnostic hand localization in cluttered industrial environments. Domain randomization is a simple technique for addressing the "reality gap" by randomly rendering unrealistic features in a simulation scene to force the neural network to learn essential domain features. We provide a new synthetic dataset for various hand detection applications in industrial environments, as well as ready-to-use pretrained instance segmentation models. To achieve robust results in a complex unstructured environment, we use multimodal input that includes both colour and depth information, which we hypothesize helps to improve the accuracy of the model prediction. In order to test this assumption, we analyze the influence of each modality and their synergy. The evaluated models were trained solely on our synthetic dataset; yet we show that our approach enables the models to outperform corresponding models trained on existing state-of-the-art datasets in terms of Average Precision and Probability-based Detection Quality.

Related papers

• Hybrid Training Approaches for LLMs: Leveraging Real and Synthetic Data to Enhance Model Performance in Domain-Specific Applications [0.0]
  This research explores a hybrid approach to fine-tuning large language models (LLMs) By leveraging a dataset combining transcribed real interactions with high-quality synthetic sessions, we aimed to overcome the limitations of domain-specific real data. The study evaluated three models: a base foundational model, a model fine-tuned with real data, and a hybrid fine-tuned model.
  arXiv  Detail & Related papers  (2024-10-11T18:16:03Z)
• Investigation of the Impact of Synthetic Training Data in the Industrial Application of Terminal Strip Object Detection [4.327763441385371]
  In this paper, we investigate the sim-to-real generalization performance of standard object detectors on the complex industrial application of terminal strip object detection. We manually annotated 300 real images of terminal strips for the evaluation. The results show the cruciality of the objects of interest to have the same scale in either domain.
  arXiv  Detail & Related papers  (2024-03-06T18:33:27Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data [78.21197488065177]
  Recent success in fine-tuning large models, that are pretrained on broad data at scale, on downstream tasks has led to a significant paradigm shift in deep learning. This paper proposes a new task-agnostic framework, textitSynBench, to measure the quality of pretrained representations using synthetic data.
  arXiv  Detail & Related papers  (2022-10-06T15:25:00Z)
• Neural-Sim: Learning to Generate Training Data with NeRF [31.81496344354997]
  We present the first fully differentiable synthetic data pipeline that uses Neural Radiance Fields (NeRFs) in a closed-loop with a target application's loss function. Our approach generates data on-demand, with no human labor, to maximize accuracy for a target task.
  arXiv  Detail & Related papers  (2022-07-22T22:48:33Z)
• Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation [117.3856882511919]
  We propose the Style-HAllucinated Dual consistEncy learning (SHADE) framework to handle domain shift. Our SHADE yields significant improvement and outperforms state-of-the-art methods by 5.07% and 8.35% on the average mIoU of three real-world datasets.
  arXiv  Detail & Related papers  (2022-04-06T02:49:06Z)
• Style-transfer GANs for bridging the domain gap in synthetic pose estimator training [8.508403388002133]
  We propose to adopt general-purpose GAN models for pixel-level image translation. The obtained models are then used either during training or inference to bridge the domain gap. Our evaluation shows a considerable improvement in model performance when compared to a model trained with the same degree of domain randomization.
  arXiv  Detail & Related papers  (2020-04-28T17:35:03Z)
• A Generative Learning Approach for Spatio-temporal Modeling in Connected Vehicular Network [55.852401381113786]
  This paper proposes LaMI (Latency Model Inpainting), a novel framework to generate a comprehensive-temporal quality framework for wireless access latency of connected vehicles. LaMI adopts the idea from image inpainting and synthesizing and can reconstruct the missing latency samples by a two-step procedure. In particular, it first discovers the spatial correlation between samples collected in various regions using a patching-based approach and then feeds the original and highly correlated samples into a Varienational Autocoder (VAE)
  arXiv  Detail & Related papers  (2020-03-16T03:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.