High-Resolution Optical Flow from 1D Attention and Correlation

• URL: http://arxiv.org/abs/2104.13918v1
• Date: Wed, 28 Apr 2021 17:56:34 GMT
• Title: High-Resolution Optical Flow from 1D Attention and Correlation
• Authors: Haofei Xu, Jiaolong Yang, Jianfei Cai, Juyong Zhang, Xin Tong
• Abstract summary: We propose a new method for high-resolution optical flow estimation with significantly less computation. We first perform a 1D attention operation in the vertical direction of the target image, and then a simple 1D correlation in the horizontal direction of the attended image. Experiments on Sintel, KITTI and real-world 4K resolution images demonstrated the effectiveness and superiority of our proposed method.
• Score: 89.61824964952949
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Optical flow is inherently a 2D search problem, and thus the computational complexity grows quadratically with respect to the search window, making large displacements matching infeasible for high-resolution images. In this paper, we propose a new method for high-resolution optical flow estimation with significantly less computation, which is achieved by factorizing 2D optical flow with 1D attention and correlation. Specifically, we first perform a 1D attention operation in the vertical direction of the target image, and then a simple 1D correlation in the horizontal direction of the attended image can achieve 2D correspondence modeling effect. The directions of attention and correlation can also be exchanged, resulting in two 3D cost volumes that are concatenated for optical flow estimation. The novel 1D formulation empowers our method to scale to very high-resolution input images while maintaining competitive performance. Extensive experiments on Sintel, KITTI and real-world 4K ($2160 \times 3840$) resolution images demonstrated the effectiveness and superiority of our proposed method.

Related papers

• Improving Robustness for Joint Optimization of Camera Poses and Decomposed Low-Rank Tensorial Radiance Fields [26.4340697184666]
  We propose an algorithm that allows joint refinement of camera pose and scene geometry represented by decomposed low-rank tensor. We also propose techniques of smoothed 2D supervision, randomly scaled kernel parameters, and edge-guided loss mask.
  arXiv  Detail & Related papers  (2024-02-20T18:59:02Z)
• Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume [6.122542233250026]
  We present MeFlow, a novel memory-efficient method for high-resolution optical flow estimation. Our method achieves competitive performance on both Sintel and KITTI benchmarks, while maintaining the highest memory efficiency on high-resolution inputs.
  arXiv  Detail & Related papers  (2023-12-06T12:43:11Z)
• UFD-PRiME: Unsupervised Joint Learning of Optical Flow and Stereo Depth through Pixel-Level Rigid Motion Estimation [4.445751695675388]
  Both optical flow and stereo disparities are image matches and can therefore benefit from joint training. We design a first network that estimates flow and disparity jointly and is trained without supervision. A second network, trained with optical flow from the first as pseudo-labels, takes disparities from the first network, estimates 3D rigid motion at every pixel, and reconstructs optical flow again.
  arXiv  Detail & Related papers  (2023-10-07T07:08:25Z)
• Dense Optical Flow from Event Cameras [55.79329250951028]
  We propose to incorporate feature correlation and sequential processing into dense optical flow estimation from event cameras. Our proposed approach computes dense optical flow and reduces the end-point error by 23% on MVSEC.
  arXiv  Detail & Related papers  (2021-08-24T07:39:08Z)
• Deep Two-View Structure-from-Motion Revisited [83.93809929963969]
  Two-view structure-from-motion (SfM) is the cornerstone of 3D reconstruction and visual SLAM. We propose to revisit the problem of deep two-view SfM by leveraging the well-posedness of the classic pipeline. Our method consists of 1) an optical flow estimation network that predicts dense correspondences between two frames; 2) a normalized pose estimation module that computes relative camera poses from the 2D optical flow correspondences, and 3) a scale-invariant depth estimation network that leverages epipolar geometry to reduce the search space, refine the dense correspondences, and estimate relative depth maps.
  arXiv  Detail & Related papers  (2021-04-01T15:31:20Z)
• Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo [61.6260594326246]
  We introduce an efficient fully-convolutional architecture that can leverage both spatial and photometric context simultaneously. Using separable 4D convolutions and 2D heat-maps reduces the size and makes more efficient.
  arXiv  Detail & Related papers  (2021-03-22T18:06:58Z)
• Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimation [109.64756528516631]
  Learning matching costs have been shown to be critical to the success of the state-of-the-art deep stereo matching methods. This paper proposes a novel solution that is able to bypass the requirement of building a 5D feature volume. Our approach achieves state-of-the-art accuracy on various datasets, and outperforms all published optical flow methods on the Sintel benchmark.
  arXiv  Detail & Related papers  (2020-10-28T09:57:00Z)
• Correlation Plenoptic Imaging between Arbitrary Planes [52.77024349608834]
  We show that the protocol enables to change the focused planes, in post-processing, and to achieve an unprecedented combination of image resolution and depth of field. Results lead the way towards the development of compact designs for correlation plenoptic imaging devices based on chaotic light, as well as high-SNR plenoptic imaging devices based on entangled photon illumination.
  arXiv  Detail & Related papers  (2020-07-23T14:26:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.