What Matters in Unsupervised Optical Flow

• URL: http://arxiv.org/abs/2006.04902v2
• Date: Fri, 14 Aug 2020 13:39:34 GMT
• Title: What Matters in Unsupervised Optical Flow
• Authors: Rico Jonschkowski, Austin Stone, Jonathan T. Barron, Ariel Gordon, Kurt Konolige, Anelia Angelova
• Abstract summary: We compare and analyze a set of key components in unsupervised optical flow. We construct a number of novel improvements to unsupervised flow models. We present a new unsupervised flow technique that significantly outperforms the previous state-of-the-art.
• Score: 51.45112526506455
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We systematically compare and analyze a set of key components in unsupervised optical flow to identify which photometric loss, occlusion handling, and smoothness regularization is most effective. Alongside this investigation we construct a number of novel improvements to unsupervised flow models, such as cost volume normalization, stopping the gradient at the occlusion mask, encouraging smoothness before upsampling the flow field, and continual self-supervision with image resizing. By combining the results of our investigation with our improved model components, we are able to present a new unsupervised flow technique that significantly outperforms the previous unsupervised state-of-the-art and performs on par with supervised FlowNet2 on the KITTI 2015 dataset, while also being significantly simpler than related approaches.

Related papers

• Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation [34.529280562470746]
  We introduce a novel self-supervised loss combining the Contrast Maximization framework with a non-linear motion prior in the form of pixel-level trajectories. Their effectiveness is demonstrated in two scenarios: In dense continuous-time motion estimation, our method improves the zero-shot performance of a synthetically trained model by 29%.
  arXiv  Detail & Related papers  (2024-07-15T15:18:28Z)
• GAFlow: Incorporating Gaussian Attention into Optical Flow [62.646389181507764]
  We push Gaussian Attention (GA) into the optical flow models to accentuate local properties during representation learning. We introduce a novel Gaussian-Constrained Layer (GCL) which can be easily plugged into existing Transformer blocks. For reliable motion analysis, we provide a new Gaussian-Guided Attention Module (GGAM)
  arXiv  Detail & Related papers  (2023-09-28T07:46:01Z)
• DistractFlow: Improving Optical Flow Estimation via Realistic Distractions and Pseudo-Labeling [49.46842536813477]
  We propose a novel data augmentation approach, DistractFlow, for training optical flow estimation models. We combine one of the frames in the pair with a distractor image depicting a similar domain, which allows for inducing visual perturbations congruent with natural objects and scenes. Our approach allows increasing the number of available training pairs significantly without requiring additional annotations.
  arXiv  Detail & Related papers  (2023-03-24T15:42:54Z)
• FG-Depth: Flow-Guided Unsupervised Monocular Depth Estimation [17.572459787107427]
  We propose a flow distillation loss to replace the typical photometric loss and a prior flow based mask to remove invalid pixels. Our approach achieves state-of-the-art results on both KITTI and NYU-Depth-v2 datasets.
  arXiv  Detail & Related papers  (2023-01-20T04:02:13Z)
• Unsupervised Flow Refinement near Motion Boundaries [3.5317804902980527]
  Unsupervised optical flow estimators based on deep learning have attracted increasing attention due to the cost and difficulty of annotating for ground truth. Our proposed algorithm detects boundaries more accurately than a baseline method with the same inputs and can improve estimates from any flow predictor without additional training.
  arXiv  Detail & Related papers  (2022-08-03T18:44:39Z)
• Self-Supervised Multi-Frame Monocular Scene Flow [61.588808225321735]
  We introduce a multi-frame monocular scene flow network based on self-supervised learning. We observe state-of-the-art accuracy among monocular scene flow methods based on self-supervised learning.
  arXiv  Detail & Related papers  (2021-05-05T17:49:55Z)
• Unsupervised Motion Representation Enhanced Network for Action Recognition [4.42249337449125]
  Motion representation between consecutive frames has proven to have great promotion to video understanding. TV-L1 method, an effective optical flow solver, is time-consuming and expensive in storage for caching the extracted optical flow. We propose UF-TSN, a novel end-to-end action recognition approach enhanced with an embedded lightweight unsupervised optical flow estimator.
  arXiv  Detail & Related papers  (2021-03-05T04:14:32Z)
• Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation [83.23707895728995]
  Unsupervised learning of optical flow has emerged as a promising alternative to supervised methods. We present a framework to use more reliable supervision from transformations. Our method consistently gets a leap of performance on several benchmarks with the best accuracy among deep unsupervised methods.
  arXiv  Detail & Related papers  (2020-03-29T14:55:24Z)
• Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization [59.9673626329892]
  We exploit the global relationship between optical flow and camera motion using epipolar geometry. We use implicit differentiation to enable back-propagation through the lower-level geometric optimization layer independent of its implementation.
  arXiv  Detail & Related papers  (2020-02-26T22:28:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.