Optical Flow in Dense Foggy Scenes using Semi-Supervised Learning

• URL: http://arxiv.org/abs/2004.01905v1
• Date: Sat, 4 Apr 2020 10:44:16 GMT
• Title: Optical Flow in Dense Foggy Scenes using Semi-Supervised Learning
• Authors: Wending Yan, Aashish Sharma, Robby T. Tan
• Abstract summary: We introduce a semi-supervised deep learning technique that employs real fog images without optical flow ground-truths in the training process. We propose a new training strategy that combines supervised synthetic-data training and unsupervised real-data training. Experimental results show that our method is effective and outperforms the state-of-the-art methods in estimating optical flow in dense foggy scenes.
• Score: 37.25344909016075
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In dense foggy scenes, existing optical flow methods are erroneous. This is due to the degradation caused by dense fog particles that break the optical flow basic assumptions such as brightness and gradient constancy. To address the problem, we introduce a semi-supervised deep learning technique that employs real fog images without optical flow ground-truths in the training process. Our network integrates the domain transformation and optical flow networks in one framework. Initially, given a pair of synthetic fog images, its corresponding clean images and optical flow ground-truths, in one training batch we train our network in a supervised manner. Subsequently, given a pair of real fog images and a pair of clean images that are not corresponding to each other (unpaired), in the next training batch, we train our network in an unsupervised manner. We then alternate the training of synthetic and real data iteratively. We use real data without ground-truths, since to have ground-truths in such conditions is intractable, and also to avoid the overfitting problem of synthetic data training, where the knowledge learned on synthetic data cannot be generalized to real data testing. Together with the network architecture design, we propose a new training strategy that combines supervised synthetic-data training and unsupervised real-data training. Experimental results show that our method is effective and outperforms the state-of-the-art methods in estimating optical flow in dense foggy scenes.

Related papers

• From Synthetic to Real: Image Dehazing Collaborating with Unlabeled Real Data [58.50411487497146]
  We propose a novel image dehazing framework collaborating with unlabeled real data. First, we develop a disentangled image dehazing network (DID-Net), which disentangles the feature representations into three component maps. Then a disentangled-consistency mean-teacher network (DMT-Net) is employed to collaborate unlabeled real data for boosting single image dehazing.
  arXiv  Detail & Related papers  (2021-08-06T04:00:28Z)
• Learning optical flow from still images [53.295332513139925]
  We introduce a framework to generate accurate ground-truth optical flow annotations quickly and in large amounts from any readily available single real picture. We virtually move the camera in the reconstructed environment with known motion vectors and rotation angles. When trained with our data, state-of-the-art optical flow networks achieve superior generalization to unseen real data.
  arXiv  Detail & Related papers  (2021-04-08T17:59:58Z)
• Optical Flow Dataset Synthesis from Unpaired Images [36.158607790844705]
  We introduce a novel method to build a training set of pseudo-real images that can be used to train optical flow in a supervised manner. Our dataset uses two unpaired frames from real data and creates pairs of frames by simulating random warps. We thus obtain the benefit of directly training on real data while having access to an exact ground truth.
  arXiv  Detail & Related papers  (2021-04-02T22:19:47Z)
• Semi-Supervised Image Deraining using Gaussian Processes [18.434430658837258]
  We propose a semi-supervised learning framework which enables the network in learning to derain using synthetic dataset. We show that the proposed method is able to effectively leverage unlabeled data thereby resulting in significantly better performance as compared to labeled-only training.
  arXiv  Detail & Related papers  (2020-09-25T17:16:16Z)
• Syn2Real Transfer Learning for Image Deraining using Gaussian Processes [92.15895515035795]
  CNN-based methods for image deraining have achieved excellent performance in terms of reconstruction error as well as visual quality. Due to challenges in obtaining real world fully-labeled image deraining datasets, existing methods are trained only on synthetically generated data. We propose a Gaussian Process-based semi-supervised learning framework which enables the network in learning to derain using synthetic dataset.
  arXiv  Detail & Related papers  (2020-06-10T00:33:18Z)
• Learning to See Through Obstructions [117.77024641706451]
  We present a learning-based approach for removing unwanted obstructions from a short sequence of images captured by a moving camera. Our method leverages the motion differences between the background and the obstructing elements to recover both layers. We show that training on synthetically generated data transfers well to real images.
  arXiv  Detail & Related papers  (2020-04-02T17:59:12Z)
• Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement [78.58603635621591]
  Training an unpaired synthetic-to-real translation network in image space is severely under-constrained. We propose a semi-supervised approach that operates on the disentangled shading and albedo layers of the image. Our two-stage pipeline first learns to predict accurate shading in a supervised fashion using physically-based renderings as targets.
  arXiv  Detail & Related papers  (2020-03-27T21:45:41Z)
• Towards Deep Unsupervised SAR Despeckling with Blind-Spot Convolutional Neural Networks [30.410981386006394]
  Deep learning techniques have outperformed classical model-based despeckling algorithms. In this paper, we propose a self-supervised Bayesian despeckling method. We show that the performance of the proposed network is very close to the supervised training approach on synthetic data and competitive on real data.
  arXiv  Detail & Related papers  (2020-01-15T12:21:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.