Imposing Consistency for Optical Flow Estimation

• URL: http://arxiv.org/abs/2204.07262v1
• Date: Thu, 14 Apr 2022 22:58:30 GMT
• Title: Imposing Consistency for Optical Flow Estimation
• Authors: Jisoo Jeong, Jamie Menjay Lin, Fatih Porikli, Nojun Kwak
• Abstract summary: Imposing consistency through proxy tasks has been shown to enhance data-driven learning. This paper introduces novel and effective consistency strategies for optical flow estimation.
• Score: 73.53204596544472
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Imposing consistency through proxy tasks has been shown to enhance data-driven learning and enable self-supervision in various tasks. This paper introduces novel and effective consistency strategies for optical flow estimation, a problem where labels from real-world data are very challenging to derive. More specifically, we propose occlusion consistency and zero forcing in the forms of self-supervised learning and transformation consistency in the form of semi-supervised learning. We apply these consistency techniques in a way that the network model learns to describe pixel-level motions better while requiring no additional annotations. We demonstrate that our consistency strategies applied to a strong baseline network model using the original datasets and labels provide further improvements, attaining the state-of-the-art results on the KITTI-2015 scene flow benchmark in the non-stereo category. Our method achieves the best foreground accuracy (4.33% in Fl-all) over both the stereo and non-stereo categories, even though using only monocular image inputs.

Related papers

• Adversarial Robustification via Text-to-Image Diffusion Models [56.37291240867549]
  Adrial robustness has been conventionally believed as a challenging property to encode for neural networks. We develop a scalable and model-agnostic solution to achieve adversarial robustness without using any data.
  arXiv  Detail & Related papers  (2024-07-26T10:49:14Z)
• OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation [55.676358801492114]
  We propose OCAI, a method that supports robust frame ambiguities by generating intermediate video frames alongside optical flows in between. Our evaluations demonstrate superior quality and enhanced optical flow accuracy on established benchmarks such as Sintel and KITTI.
  arXiv  Detail & Related papers  (2024-03-26T20:23:48Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• Consistency Regularization for Domain Adaptation [17.067348505083327]
  Unsupervised domain adaptation (UDA) tries to solve this problem by studying how more accessible data can be used to train and adapt models to real world images. Recent UDA methods applies self-learning by training on pixel-wise classification loss using a student and teacher network. We propose the addition of a consistency regularization term to semi-supervised UDA by modelling the inter-pixel relationship between elements in networks' output.
  arXiv  Detail & Related papers  (2022-08-23T17:07:06Z)
• Semi-Supervised Learning of Optical Flow by Flow Supervisor [16.406213579356795]
  We propose a practical fine tuning method to adapt a pretrained model to a target dataset without ground truth flows. This design is aimed at stable convergence and better accuracy over conventional self-supervision methods. We achieve meaningful improvements over state-of-the-art optical flow models on Sintel and KITTI benchmarks.
  arXiv  Detail & Related papers  (2022-07-21T06:11:52Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)
• Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation [71.76008290101214]
  DistillFlow is a knowledge distillation approach to learning optical flow. It achieves state-of-the-art unsupervised learning performance on both KITTI and Sintel datasets. Our models ranked 1st among all monocular methods on the KITTI 2015 benchmark, and outperform all published methods on the Sintel Final benchmark.
  arXiv  Detail & Related papers  (2021-06-08T09:13:34Z)
• Federated Self-Supervised Learning of Multi-Sensor Representations for Embedded Intelligence [8.110949636804772]
  Smartphones, wearables, and Internet of Things (IoT) devices produce a wealth of data that cannot be accumulated in a centralized repository for learning supervised models. We propose a self-supervised approach termed textitscalogram-signal correspondence learning based on wavelet transform to learn useful representations from unlabeled sensor inputs. We extensively assess the quality of learned features with our multi-view strategy on diverse public datasets, achieving strong performance in all domains.
  arXiv  Detail & Related papers  (2020-07-25T21:59:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.