DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework

• URL: http://arxiv.org/abs/2503.14880v1
• Date: Wed, 19 Mar 2025 04:18:18 GMT
• Title: DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework
• Authors: Henrique Morimitsu, Xiaobin Zhu, Roberto M. Cesar Jr., Xiangyang Ji, Xu-Cheng Yin,
• Abstract summary: We propose DPFlow, an adaptive optical flow architecture capable of generalizing up to 8K resolution inputs.<n>We also introduce Kubric-NK, a new benchmark for evaluating optical flow methods with input resolutions ranging from 1K to 8K.
• Score: 57.69159159559054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical flow estimation is essential for video processing tasks, such as restoration and action recognition. The quality of videos is constantly increasing, with current standards reaching 8K resolution. However, optical flow methods are usually designed for low resolution and do not generalize to large inputs due to their rigid architectures. They adopt downscaling or input tiling to reduce the input size, causing a loss of details and global information. There is also a lack of optical flow benchmarks to judge the actual performance of existing methods on high-resolution samples. Previous works only conducted qualitative high-resolution evaluations on hand-picked samples. This paper fills this gap in optical flow estimation in two ways. We propose DPFlow, an adaptive optical flow architecture capable of generalizing up to 8K resolution inputs while trained with only low-resolution samples. We also introduce Kubric-NK, a new benchmark for evaluating optical flow methods with input resolutions ranging from 1K to 8K. Our high-resolution evaluation pushes the boundaries of existing methods and reveals new insights about their generalization capabilities. Extensive experimental results show that DPFlow achieves state-of-the-art results on the MPI-Sintel, KITTI 2015, Spring, and other high-resolution benchmarks.

Related papers

• Benchmarking the Robustness of Optical Flow Estimation to Corruptions [25.789811424859554]
  We introduce 7 temporal corruptions specifically designed for the benchmarking of optical flow models. Two robustness benchmarks, KITTI-FC and GoPro-FC, are established as the first corruption benchmark for optical flow estimation. 29 model variants from 15 optical flow methods are evaluated, yielding 10 intriguing observations.
  arXiv  Detail & Related papers  (2024-11-22T11:31:01Z)
• NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices [6.157420789049589]
  We propose a highly efficient optical flow method that balances high accuracy with reduced computational demands. We introduce new components including a much more light-weight backbone and a fast refinement module. Our model achieves a 10x-70x speedup while maintaining comparable performance on both synthetic and real-world data.
  arXiv  Detail & Related papers  (2024-08-19T17:13:34Z)
• MemFlow: Optical Flow Estimation and Prediction with Memory [54.22820729477756]
  We present MemFlow, a real-time method for optical flow estimation and prediction with memory. Our method enables memory read-out and update modules for aggregating historical motion information in real-time. Our approach seamlessly extends to the future prediction of optical flow based on past observations.
  arXiv  Detail & Related papers  (2024-04-07T04:56:58Z)
• AnyFlow: Arbitrary Scale Optical Flow with Implicit Neural Representation [17.501820140334328]
  We introduce AnyFlow, a robust network that estimates accurate flow from images of various resolutions. We establish a new state-of-the-art performance of cross-dataset generalization on the KITTI dataset.
  arXiv  Detail & Related papers  (2023-03-29T07:03:51Z)
• Rethinking Optical Flow from Geometric Matching Consistent Perspective [38.014569953980754]
  We propose a rethinking to previous optical flow estimation. We use GIM as a pre-training task for the optical flow estimation (MatchFlow) with better feature representations. Our method achieves 11.5% and 10.1% error reduction from GMA on Sintel clean pass and KITTI test set.
  arXiv  Detail & Related papers  (2023-03-15T06:00:38Z)
• Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method [51.30748775681917]
  We consider the task of low-light image enhancement (LLIE) and introduce a large-scale database consisting of images at 4K and 8K resolution. We conduct systematic benchmarking studies and provide a comparison of current LLIE algorithms. As a second contribution, we introduce LLFormer, a transformer-based low-light enhancement method.
  arXiv  Detail & Related papers  (2022-12-22T09:05:07Z)
• Deep Equilibrium Optical Flow Estimation [80.80992684796566]
  Recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms. These RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation. We propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer.
  arXiv  Detail & Related papers  (2022-04-18T17:53:44Z)
• DIP: Deep Inverse Patchmatch for High-Resolution Optical Flow [7.73554718719193]
  We propose a novel Patchmatch-based framework to work on high-resolution optical flow estimation. It can get high-precision results with lower memory benefiting from propagation and local search of Patchmatch. Our method ranks first on all the metrics on the popular KITTI2015 benchmark, and ranks second on EPE on the Sintel clean benchmark among published optical flow methods.
  arXiv  Detail & Related papers  (2022-04-01T10:13:59Z)
• GMFlow: Learning Optical Flow via Global Matching [124.57850500778277]
  We propose a GMFlow framework for learning optical flow estimation. It consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. Our new framework outperforms 32-iteration RAFT's performance on the challenging Sintel benchmark.
  arXiv  Detail & Related papers  (2021-11-26T18:59:56Z)
• Dense Optical Flow from Event Cameras [55.79329250951028]
  We propose to incorporate feature correlation and sequential processing into dense optical flow estimation from event cameras. Our proposed approach computes dense optical flow and reduces the end-point error by 23% on MVSEC.
  arXiv  Detail & Related papers  (2021-08-24T07:39:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.