Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume

• URL: http://arxiv.org/abs/2312.03790v1
• Date: Wed, 6 Dec 2023 12:43:11 GMT
• Title: Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume
• Authors: Gangwei Xu, Shujun Chen, Hao Jia, Miaojie Feng, Xin Yang
• Abstract summary: We present MeFlow, a novel memory-efficient method for high-resolution optical flow estimation. Our method achieves competitive performance on both Sintel and KITTI benchmarks, while maintaining the highest memory efficiency on high-resolution inputs.
• Score: 6.122542233250026
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The full 4D cost volume in Recurrent All-Pairs Field Transforms (RAFT) or global matching by Transformer achieves impressive performance for optical flow estimation. However, their memory consumption increases quadratically with input resolution, rendering them impractical for high-resolution images. In this paper, we present MeFlow, a novel memory-efficient method for high-resolution optical flow estimation. The key of MeFlow is a recurrent local orthogonal cost volume representation, which decomposes the 2D search space dynamically into two 1D orthogonal spaces, enabling our method to scale effectively to very high-resolution inputs. To preserve essential information in the orthogonal space, we utilize self attention to propagate feature information from the 2D space to the orthogonal space. We further propose a radius-distribution multi-scale lookup strategy to model the correspondences of large displacements at a negligible cost. We verify the efficiency and effectiveness of our method on the challenging Sintel and KITTI benchmarks, and real-world 4K ($2160\!\times\!3840$) images. Our method achieves competitive performance on both Sintel and KITTI benchmarks, while maintaining the highest memory efficiency on high-resolution inputs.

Related papers

• Hybrid Cost Volume for Memory-Efficient Optical Flow [10.760762249786344]
  Current state-of-the-art flow methods are mostly based on dense all-pairs cost volumes. We propose a novel Hybrid Cost Volume for memory-efficient optical flow, named HCV. Based on HCV, we design a memory-efficient optical flow network, named HCVFlow.
  arXiv  Detail & Related papers  (2024-09-06T12:49:34Z)
• Mamba-based Light Field Super-Resolution with Efficient Subspace Scanning [48.99361249764921]
  Transformer-based methods have demonstrated impressive performance in 4D light field (LF) super-resolution. However, their quadratic complexity hinders the efficient processing of high resolution 4D inputs. We propose a Mamba-based Light Field Super-Resolution method, named MLFSR, by designing an efficient subspace scanning strategy.
  arXiv  Detail & Related papers  (2024-06-23T11:28:08Z)
• DIFT: Dynamic Iterative Field Transforms for Memory Efficient Optical Flow [44.57023882737517]
  We introduce a lightweight low-latency and memory-efficient model for optical flow estimation. DIFT is feasible for edge applications such as mobile, XR, micro UAVs, robotics and cameras. We demonstrate first real-time cost-volume-based optical flow DL architecture on Snapdragon 8 Gen 1 HTP efficient mobile AI accelerator.
  arXiv  Detail & Related papers  (2023-06-09T06:10:59Z)
• SALISA: Saliency-based Input Sampling for Efficient Video Object Detection [58.22508131162269]
  We propose SALISA, a novel non-uniform SALiency-based Input SAmpling technique for video object detection. We show that SALISA significantly improves the detection of small objects.
  arXiv  Detail & Related papers  (2022-04-05T17:59:51Z)
• High-Resolution Optical Flow from 1D Attention and Correlation [89.61824964952949]
  We propose a new method for high-resolution optical flow estimation with significantly less computation. We first perform a 1D attention operation in the vertical direction of the target image, and then a simple 1D correlation in the horizontal direction of the attended image. Experiments on Sintel, KITTI and real-world 4K resolution images demonstrated the effectiveness and superiority of our proposed method.
  arXiv  Detail & Related papers  (2021-04-28T17:56:34Z)
• Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo [61.6260594326246]
  We introduce an efficient fully-convolutional architecture that can leverage both spatial and photometric context simultaneously. Using separable 4D convolutions and 2D heat-maps reduces the size and makes more efficient.
  arXiv  Detail & Related papers  (2021-03-22T18:06:58Z)
• Displacement-Invariant Cost Computation for Efficient Stereo Matching [122.94051630000934]
  Deep learning methods have dominated stereo matching leaderboards by yielding unprecedented disparity accuracy. But their inference time is typically slow, on the order of seconds for a pair of 540p images. We propose a emphdisplacement-invariant cost module to compute the matching costs without needing a 4D feature volume.
  arXiv  Detail & Related papers  (2020-12-01T23:58:16Z)
• Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimation [109.64756528516631]
  Learning matching costs have been shown to be critical to the success of the state-of-the-art deep stereo matching methods. This paper proposes a novel solution that is able to bypass the requirement of building a 5D feature volume. Our approach achieves state-of-the-art accuracy on various datasets, and outperforms all published optical flow methods on the Sintel benchmark.
  arXiv  Detail & Related papers  (2020-10-28T09:57:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.