Optical Flow Estimation in the Deep Learning Age

• URL: http://arxiv.org/abs/2004.02853v1
• Date: Mon, 6 Apr 2020 17:45:43 GMT
• Title: Optical Flow Estimation in the Deep Learning Age
• Authors: Junhwa Hur, Stefan Roth
• Abstract summary: We review the developments from early work to the current state of CNNs for optical flow estimation. We discuss some of their technical details and compare them to recapitulate which technical contribution led to the most significant accuracy improvements. We provide an overview of the various optical flow approaches introduced in the deep learning age.
• Score: 27.477810324117016
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Akin to many subareas of computer vision, the recent advances in deep learning have also significantly influenced the literature on optical flow. Previously, the literature had been dominated by classical energy-based models, which formulate optical flow estimation as an energy minimization problem. However, as the practical benefits of Convolutional Neural Networks (CNNs) over conventional methods have become apparent in numerous areas of computer vision and beyond, they have also seen increased adoption in the context of motion estimation to the point where the current state of the art in terms of accuracy is set by CNN approaches. We first review this transition as well as the developments from early work to the current state of CNNs for optical flow estimation. Alongside, we discuss some of their technical details and compare them to recapitulate which technical contribution led to the most significant accuracy improvements. Then we provide an overview of the various optical flow approaches introduced in the deep learning age, including those based on alternative learning paradigms (e.g., unsupervised and semi-supervised methods) as well as the extension to the multi-frame case, which is able to yield further accuracy improvements.

Related papers

• Robust Optical Flow Computation: A Higher-Order Differential Approach [0.0]
  This research proposes an innovative algorithm for optical flow computation, utilizing the higher precision of second-order Taylor series approximation. An impressive showcase of the algorithm's capabilities emerges through its performance on optical flow benchmarks such as KITTI and Middlebury.
  arXiv  Detail & Related papers  (2024-10-12T15:20:11Z)
• Skin the sheep not only once: Reusing Various Depth Datasets to Drive the Learning of Optical Flow [25.23550076996421]
  We propose to leverage the geometric connection between optical flow estimation and stereo matching. We turn the monocular depth datasets into stereo ones via virtual disparity. We also introduce virtual camera motion into stereo data to produce additional flows along the vertical direction.
  arXiv  Detail & Related papers  (2023-10-03T06:56:07Z)
• Optical Flow for Autonomous Driving: Applications, Challenges and Improvements [0.9023847175654602]
  We propose and evaluate training strategies to improve a learning-based optical flow algorithm. While trained with synthetic data, the model demonstrates strong capabilities to generalize to real world fisheye data. We propose a novel, generic semi-supervised framework that significantly boosts performances of existing methods in low light.
  arXiv  Detail & Related papers  (2023-01-11T12:01:42Z)
• Deep 360$^\circ$ Optical Flow Estimation Based on Multi-Projection Fusion [10.603670927163002]
  This paper focuses on the 360$circ$ optical flow estimation using deep neural networks to support increasingly popular VR applications. We propose a novel multi-projection fusion framework that fuses the optical flow predicted by the models trained using different projection methods. We also build the first large-scale panoramic optical flow dataset to support the training of neural networks and the evaluation of panoramic optical flow estimation methods.
  arXiv  Detail & Related papers  (2022-07-27T16:48:32Z)
• Wasserstein Flow Meets Replicator Dynamics: A Mean-Field Analysis of Representation Learning in Actor-Critic [137.04558017227583]
  Actor-critic (AC) algorithms, empowered by neural networks, have had significant empirical success in recent years. We take a mean-field perspective on the evolution and convergence of feature-based neural AC. We prove that neural AC finds the globally optimal policy at a sublinear rate.
  arXiv  Detail & Related papers  (2021-12-27T06:09:50Z)
• Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation [71.76008290101214]
  DistillFlow is a knowledge distillation approach to learning optical flow. It achieves state-of-the-art unsupervised learning performance on both KITTI and Sintel datasets. Our models ranked 1st among all monocular methods on the KITTI 2015 benchmark, and outperform all published methods on the Sintel Final benchmark.
  arXiv  Detail & Related papers  (2021-06-08T09:13:34Z)
• PCA Event-Based Otical Flow for Visual Odometry [0.0]
  We present a Principal Component Analysis approach to the problem of event-based optical flow estimation. We show that the best variant of our proposed method, dedicated to the real-time context of visual odometry, is about two times faster compared to state-of-the-art implementations.
  arXiv  Detail & Related papers  (2021-05-08T18:30:44Z)
• An Ode to an ODE [78.97367880223254]
  We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the group O(d) This nested system of two flows provides stability and effectiveness of training and provably solves the gradient vanishing-explosion problem.
  arXiv  Detail & Related papers  (2020-06-19T22:05:19Z)
• Cascaded Deep Video Deblurring Using Temporal Sharpness Prior [88.98348546566675]
  The proposed algorithm mainly consists of optical flow estimation from intermediate latent frames and latent frame restoration steps. It first develops a deep CNN model to estimate optical flow from intermediate latent frames and then restores the latent frames based on the estimated optical flow. We show that exploring the domain knowledge of video deblurring is able to make the deep CNN model more compact and efficient.
  arXiv  Detail & Related papers  (2020-04-06T09:13:49Z)
• Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization [59.9673626329892]
  We exploit the global relationship between optical flow and camera motion using epipolar geometry. We use implicit differentiation to enable back-propagation through the lower-level geometric optimization layer independent of its implementation.
  arXiv  Detail & Related papers  (2020-02-26T22:28:00Z)
• Large Batch Training Does Not Need Warmup [111.07680619360528]
  Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications. In this paper, we propose a novel Complete Layer-wise Adaptive Rate Scaling (CLARS) algorithm for large-batch training. Based on our analysis, we bridge the gap and illustrate the theoretical insights for three popular large-batch training techniques.
  arXiv  Detail & Related papers  (2020-02-04T23:03:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.