PennSyn2Real: Training Object Recognition Models without Human Labeling
- URL: http://arxiv.org/abs/2009.10292v2
- Date: Fri, 16 Oct 2020 04:58:40 GMT
- Title: PennSyn2Real: Training Object Recognition Models without Human Labeling
- Authors: Ty Nguyen, Ian D. Miller, Avi Cohen, Dinesh Thakur, Shashank Prasad,
Camillo J. Taylor, Pratik Chaudrahi, Vijay Kumar
- Abstract summary: We propose PennSyn2Real - a synthetic dataset consisting of more than 100,000 4K images of more than 20 types of micro aerial vehicles (MAVs)
The dataset can be used to generate arbitrary numbers of training images for high-level computer vision tasks such as MAV detection and classification.
We show that synthetic data generated using this framework can be directly used to train CNN models for common object recognition tasks such as detection and segmentation.
- Score: 12.923677573437699
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Scalable training data generation is a critical problem in deep learning. We
propose PennSyn2Real - a photo-realistic synthetic dataset consisting of more
than 100,000 4K images of more than 20 types of micro aerial vehicles (MAVs).
The dataset can be used to generate arbitrary numbers of training images for
high-level computer vision tasks such as MAV detection and classification. Our
data generation framework bootstraps chroma-keying, a mature cinematography
technique with a motion tracking system, providing artifact-free and curated
annotated images where object orientations and lighting are controlled. This
framework is easy to set up and can be applied to a broad range of objects,
reducing the gap between synthetic and real-world data. We show that synthetic
data generated using this framework can be directly used to train CNN models
for common object recognition tasks such as detection and segmentation. We
demonstrate competitive performance in comparison with training using only real
images. Furthermore, bootstrapping the generated synthetic data in few-shot
learning can significantly improve the overall performance, reducing the number
of required training data samples to achieve the desired accuracy.
Related papers
- Analysis of Classifier Training on Synthetic Data for Cross-Domain Datasets [4.696575161583618]
This study focuses on camera-based traffic sign recognition applications for advanced driver assistance systems and autonomous driving.
The proposed augmentation pipeline of synthetic datasets includes novel augmentation processes such as structured shadows and gaussian specular highlights.
Experiments showed that a synthetic image-based approach outperforms in most cases real image-based training when applied to cross-domain test datasets.
arXiv Detail & Related papers (2024-10-30T07:11:41Z) - DataDream: Few-shot Guided Dataset Generation [90.09164461462365]
We propose a framework for synthesizing classification datasets that more faithfully represents the real data distribution.
DataDream fine-tunes LoRA weights for the image generation model on the few real images before generating the training data using the adapted model.
We then fine-tune LoRA weights for CLIP using the synthetic data to improve downstream image classification over previous approaches on a large variety of datasets.
arXiv Detail & Related papers (2024-07-15T17:10:31Z) - Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate.
We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data.
Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
arXiv Detail & Related papers (2024-03-23T22:32:06Z) - Image Captions are Natural Prompts for Text-to-Image Models [70.30915140413383]
We analyze the relationship between the training effect of synthetic data and the synthetic data distribution induced by prompts.
We propose a simple yet effective method that prompts text-to-image generative models to synthesize more informative and diverse training data.
Our method significantly improves the performance of models trained on synthetic training data.
arXiv Detail & Related papers (2023-07-17T14:38:11Z) - Synthetic Image Data for Deep Learning [0.294944680995069]
Realistic synthetic image data rendered from 3D models can be used to augment image sets and train image classification semantic segmentation models.
We show how high quality physically-based rendering and domain randomization can efficiently create a large synthetic dataset based on production 3D CAD models of a real vehicle.
arXiv Detail & Related papers (2022-12-12T20:28:13Z) - Synthetic Data for Object Classification in Industrial Applications [53.180678723280145]
In object classification, capturing a large number of images per object and in different conditions is not always possible.
This work explores the creation of artificial images using a game engine to cope with limited data in the training dataset.
arXiv Detail & Related papers (2022-12-09T11:43:04Z) - Is synthetic data from generative models ready for image recognition? [69.42645602062024]
We study whether and how synthetic images generated from state-of-the-art text-to-image generation models can be used for image recognition tasks.
We showcase the powerfulness and shortcomings of synthetic data from existing generative models, and propose strategies for better applying synthetic data for recognition tasks.
arXiv Detail & Related papers (2022-10-14T06:54:24Z) - BIM Hyperreality: Data Synthesis Using BIM and Hyperrealistic Rendering
for Deep Learning [3.4461633417989184]
We present a concept of a hybrid system for training a neural network for building object recognition in photos.
For the specific case study presented in this paper, our results show that a neural network trained with synthetic data can be used to identify building objects from photos without using photos in the training data.
arXiv Detail & Related papers (2021-05-10T04:08:24Z) - Syn2Real Transfer Learning for Image Deraining using Gaussian Processes [92.15895515035795]
CNN-based methods for image deraining have achieved excellent performance in terms of reconstruction error as well as visual quality.
Due to challenges in obtaining real world fully-labeled image deraining datasets, existing methods are trained only on synthetically generated data.
We propose a Gaussian Process-based semi-supervised learning framework which enables the network in learning to derain using synthetic dataset.
arXiv Detail & Related papers (2020-06-10T00:33:18Z) - Can Synthetic Data Improve Object Detection Results for Remote Sensing
Images? [15.466412729455874]
We propose the use of realistic synthetic data with a wide distribution to improve the performance of remote sensing image aircraft detection.
We randomly set the parameters during rendering, such as the size of the instance and the class of background images.
In order to make the synthetic images more realistic, we refine the synthetic images at the pixel level using CycleGAN with real unlabeled images.
arXiv Detail & Related papers (2020-06-09T02:23:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.