Optical Flow Dataset Synthesis from Unpaired Images
- URL: http://arxiv.org/abs/2104.02615v1
- Date: Fri, 2 Apr 2021 22:19:47 GMT
- Title: Optical Flow Dataset Synthesis from Unpaired Images
- Authors: Adrian W\"alchli and Paolo Favaro
- Abstract summary: We introduce a novel method to build a training set of pseudo-real images that can be used to train optical flow in a supervised manner.
Our dataset uses two unpaired frames from real data and creates pairs of frames by simulating random warps.
We thus obtain the benefit of directly training on real data while having access to an exact ground truth.
- Score: 36.158607790844705
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The estimation of optical flow is an ambiguous task due to the lack of
correspondence at occlusions, shadows, reflections, lack of texture and changes
in illumination over time. Thus, unsupervised methods face major challenges as
they need to tune complex cost functions with several terms designed to handle
each of these sources of ambiguity. In contrast, supervised methods avoid these
challenges altogether by relying on explicit ground truth optical flow obtained
directly from synthetic or real data. In the case of synthetic data, the ground
truth provides an exact and explicit description of what optical flow to assign
to a given scene. However, the domain gap between synthetic data and real data
often limits the ability of a trained network to generalize. In the case of
real data, the ground truth is obtained through multiple sensors and additional
data processing, which might introduce persistent errors and contaminate it. As
a solution to these issues, we introduce a novel method to build a training set
of pseudo-real images that can be used to train optical flow in a supervised
manner. Our dataset uses two unpaired frames from real data and creates pairs
of frames by simulating random warps, occlusions with super-pixels, shadows and
illumination changes, and associates them to their corresponding exact optical
flow. We thus obtain the benefit of directly training on real data while having
access to an exact ground truth. Training with our datasets on the Sintel and
KITTI benchmarks is straightforward and yields models on par or with state of
the art performance compared to much more sophisticated training approaches.
Related papers
- The Devil in the Details: Simple and Effective Optical Flow Synthetic
Data Generation [19.945859289278534]
We show that the required characteristics in an optical flow dataset are rather simple and present a simpler synthetic data generation method.
With 2D motion-based datasets, we systematically analyze the simplest yet critical factors for generating synthetic datasets.
arXiv Detail & Related papers (2023-08-14T18:01:45Z) - A New Benchmark: On the Utility of Synthetic Data with Blender for Bare
Supervised Learning and Downstream Domain Adaptation [42.2398858786125]
Deep learning in computer vision has achieved great success with the price of large-scale labeled training data.
The uncontrollable data collection process produces non-IID training and test data, where undesired duplication may exist.
To circumvent them, an alternative is to generate synthetic data via 3D rendering with domain randomization.
arXiv Detail & Related papers (2023-03-16T09:03:52Z) - Imposing Consistency for Optical Flow Estimation [73.53204596544472]
Imposing consistency through proxy tasks has been shown to enhance data-driven learning.
This paper introduces novel and effective consistency strategies for optical flow estimation.
arXiv Detail & Related papers (2022-04-14T22:58:30Z) - Occlusion-aware Unsupervised Learning of Depth from 4-D Light Fields [50.435129905215284]
We present an unsupervised learning-based depth estimation method for 4-D light field processing and analysis.
Based on the basic knowledge of the unique geometry structure of light field data, we explore the angular coherence among subsets of the light field views to estimate depth maps.
Our method can significantly shrink the performance gap between the previous unsupervised method and supervised ones, and produce depth maps with comparable accuracy to traditional methods with obviously reduced computational cost.
arXiv Detail & Related papers (2021-06-06T06:19:50Z) - Learning optical flow from still images [53.295332513139925]
We introduce a framework to generate accurate ground-truth optical flow annotations quickly and in large amounts from any readily available single real picture.
We virtually move the camera in the reconstructed environment with known motion vectors and rotation angles.
When trained with our data, state-of-the-art optical flow networks achieve superior generalization to unseen real data.
arXiv Detail & Related papers (2021-04-08T17:59:58Z) - DeFlow: Learning Complex Image Degradations from Unpaired Data with
Conditional Flows [145.83812019515818]
We propose DeFlow, a method for learning image degradations from unpaired data.
We model the degradation process in the latent space of a shared flow-decoder network.
We validate our DeFlow formulation on the task of joint image restoration and super-resolution.
arXiv Detail & Related papers (2021-01-14T18:58:01Z) - Deep traffic light detection by overlaying synthetic context on
arbitrary natural images [49.592798832978296]
We propose a method to generate artificial traffic-related training data for deep traffic light detectors.
This data is generated using basic non-realistic computer graphics to blend fake traffic scenes on top of arbitrary image backgrounds.
It also tackles the intrinsic data imbalance problem in traffic light datasets, caused mainly by the low amount of samples of the yellow state.
arXiv Detail & Related papers (2020-11-07T19:57:22Z) - Syn2Real Transfer Learning for Image Deraining using Gaussian Processes [92.15895515035795]
CNN-based methods for image deraining have achieved excellent performance in terms of reconstruction error as well as visual quality.
Due to challenges in obtaining real world fully-labeled image deraining datasets, existing methods are trained only on synthetically generated data.
We propose a Gaussian Process-based semi-supervised learning framework which enables the network in learning to derain using synthetic dataset.
arXiv Detail & Related papers (2020-06-10T00:33:18Z) - Optical Flow in Dense Foggy Scenes using Semi-Supervised Learning [37.25344909016075]
We introduce a semi-supervised deep learning technique that employs real fog images without optical flow ground-truths in the training process.
We propose a new training strategy that combines supervised synthetic-data training and unsupervised real-data training.
Experimental results show that our method is effective and outperforms the state-of-the-art methods in estimating optical flow in dense foggy scenes.
arXiv Detail & Related papers (2020-04-04T10:44:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.