AI Playground: Unreal Engine-based Data Ablation Tool for Deep Learning
- URL: http://arxiv.org/abs/2007.06153v1
- Date: Mon, 13 Jul 2020 02:04:39 GMT
- Title: AI Playground: Unreal Engine-based Data Ablation Tool for Deep Learning
- Authors: Mehdi Mousavi, Aashis Khanal, Rolando Estrada
- Abstract summary: Machine learning requires data, but acquiring and labeling real-world data is challenging, expensive, and time-consuming.
We present AI Playground (AIP), an open-source, Unreal Engine-based tool for generating and labeling virtual image data.
We trained deep neural networks to predict depth values, surface normals, or object labels and assessed each network's intra- and cross-dataset performance.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Machine learning requires data, but acquiring and labeling real-world data is
challenging, expensive, and time-consuming. More importantly, it is nearly
impossible to alter real data post-acquisition (e.g., change the illumination
of a room), making it very difficult to measure how specific properties of the
data affect performance. In this paper, we present AI Playground (AIP), an
open-source, Unreal Engine-based tool for generating and labeling virtual image
data. With AIP, it is trivial to capture the same image under different
conditions (e.g., fidelity, lighting, etc.) and with different ground truths
(e.g., depth or surface normal values). AIP is easily extendable and can be
used with or without code. To validate our proposed tool, we generated eight
datasets of otherwise identical but varying lighting and fidelity conditions.
We then trained deep neural networks to predict (1) depth values, (2) surface
normals, or (3) object labels and assessed each network's intra- and
cross-dataset performance. Among other insights, we verified that sensitivity
to different settings is problem-dependent. We confirmed the findings of other
studies that segmentation models are very sensitive to fidelity, but we also
found that they are just as sensitive to lighting. In contrast, depth and
normal estimation models seem to be less sensitive to fidelity or lighting and
more sensitive to the structure of the image. Finally, we tested our trained
depth-estimation networks on two real-world datasets and obtained results
comparable to training on real data alone, confirming that our virtual
environments are realistic enough for real-world tasks.
Related papers
- Object Detector Differences when using Synthetic and Real Training Data [0.0]
We train the YOLOv3 object detector on real and synthetic images from city environments.
We perform a similarity analysis using Centered Kernel Alignment (CKA) to explore the effects of training on synthetic data on a layer-wise basis.
The results show that the largest similarity between a detector trained on real data and a detector trained on synthetic data was in the early layers, and the largest difference was in the head part.
arXiv Detail & Related papers (2023-12-01T16:27:48Z) - A New Benchmark: On the Utility of Synthetic Data with Blender for Bare
Supervised Learning and Downstream Domain Adaptation [42.2398858786125]
Deep learning in computer vision has achieved great success with the price of large-scale labeled training data.
The uncontrollable data collection process produces non-IID training and test data, where undesired duplication may exist.
To circumvent them, an alternative is to generate synthetic data via 3D rendering with domain randomization.
arXiv Detail & Related papers (2023-03-16T09:03:52Z) - Synthetic Data for Object Classification in Industrial Applications [53.180678723280145]
In object classification, capturing a large number of images per object and in different conditions is not always possible.
This work explores the creation of artificial images using a game engine to cope with limited data in the training dataset.
arXiv Detail & Related papers (2022-12-09T11:43:04Z) - Learning to Simulate Realistic LiDARs [66.7519667383175]
We introduce a pipeline for data-driven simulation of a realistic LiDAR sensor.
We show that our model can learn to encode realistic effects such as dropped points on transparent surfaces.
We use our technique to learn models of two distinct LiDAR sensors and use them to improve simulated LiDAR data accordingly.
arXiv Detail & Related papers (2022-09-22T13:12:54Z) - Towards Scale Consistent Monocular Visual Odometry by Learning from the
Virtual World [83.36195426897768]
We propose VRVO, a novel framework for retrieving the absolute scale from virtual data.
We first train a scale-aware disparity network using both monocular real images and stereo virtual data.
The resulting scale-consistent disparities are then integrated with a direct VO system.
arXiv Detail & Related papers (2022-03-11T01:51:54Z) - Towards Optimal Strategies for Training Self-Driving Perception Models
in Simulation [98.51313127382937]
We focus on the use of labels in the synthetic domain alone.
Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator.
We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
arXiv Detail & Related papers (2021-11-15T18:37:43Z) - Improving Deep Stereo Network Generalization with Geometric Priors [93.09496073476275]
Large datasets of diverse real-world scenes with dense ground truth are difficult to obtain.
Many algorithms rely on small real-world datasets of similar scenes or synthetic datasets.
We propose to incorporate prior knowledge of scene geometry into an end-to-end stereo network to help networks generalize better.
arXiv Detail & Related papers (2020-08-25T15:24:02Z) - Learning to simulate complex scenes [18.51564016785853]
This paper explores content adaptation in the context of semantic segmentation.
We propose a scalable discretization-and-relaxation (SDR) approach to optimize the attribute values and obtain a training set of similar content to real-world data.
Experiment shows our system can generate reasonable and useful scenes, from which we obtain promising real-world segmentation accuracy.
arXiv Detail & Related papers (2020-06-25T17:51:34Z) - Syn2Real Transfer Learning for Image Deraining using Gaussian Processes [92.15895515035795]
CNN-based methods for image deraining have achieved excellent performance in terms of reconstruction error as well as visual quality.
Due to challenges in obtaining real world fully-labeled image deraining datasets, existing methods are trained only on synthetically generated data.
We propose a Gaussian Process-based semi-supervised learning framework which enables the network in learning to derain using synthetic dataset.
arXiv Detail & Related papers (2020-06-10T00:33:18Z) - Exploring the Capabilities and Limits of 3D Monocular Object Detection
-- A Study on Simulation and Real World Data [0.0]
3D object detection based on monocular camera data is key enabler for autonomous driving.
Recent deep learning methods show promising results to recover depth information from single images.
In this paper, we evaluate the performance of a 3D object detection pipeline which is parameterizable with different depth estimation configurations.
arXiv Detail & Related papers (2020-05-15T09:05:17Z) - Virtual to Real adaptation of Pedestrian Detectors [9.432150710329607]
ViPeD is a new synthetically generated set of images collected with the graphical engine of the video game GTA V - Grand Theft Auto V.
We propose two different Domain Adaptation techniques suitable for the pedestrian detection task, but possibly applicable to general object detection.
Experiments show that the network trained with ViPeD can generalize over unseen real-world scenarios better than the detector trained over real-world data.
arXiv Detail & Related papers (2020-01-09T14:50:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.