Rethinking Optical Flow from Geometric Matching Consistent Perspective
- URL: http://arxiv.org/abs/2303.08384v1
- Date: Wed, 15 Mar 2023 06:00:38 GMT
- Title: Rethinking Optical Flow from Geometric Matching Consistent Perspective
- Authors: Qiaole Dong and Chenjie Cao and Yanwei Fu
- Abstract summary: We propose a rethinking to previous optical flow estimation.
We use GIM as a pre-training task for the optical flow estimation (MatchFlow) with better feature representations.
Our method achieves 11.5% and 10.1% error reduction from GMA on Sintel clean pass and KITTI test set.
- Score: 38.014569953980754
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Optical flow estimation is a challenging problem remaining unsolved. Recent
deep learning based optical flow models have achieved considerable success.
However, these models often train networks from the scratch on standard optical
flow data, which restricts their ability to robustly and geometrically match
image features. In this paper, we propose a rethinking to previous optical flow
estimation. We particularly leverage Geometric Image Matching (GIM) as a
pre-training task for the optical flow estimation (MatchFlow) with better
feature representations, as GIM shares some common challenges as optical flow
estimation, and with massive labeled real-world data. Thus, matching static
scenes helps to learn more fundamental feature correlations of objects and
scenes with consistent displacements. Specifically, the proposed MatchFlow
model employs a QuadTree attention-based network pre-trained on MegaDepth to
extract coarse features for further flow regression. Extensive experiments show
that our model has great cross-dataset generalization. Our method achieves
11.5% and 10.1% error reduction from GMA on Sintel clean pass and KITTI test
set. At the time of anonymous submission, our MatchFlow(G) enjoys
state-of-the-art performance on Sintel clean and final pass compared to
published approaches with comparable computation and memory footprint. Codes
and models will be released in https://github.com/DQiaole/MatchFlow.
Related papers
- HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment [0.5825410941577593]
We present a novel method, dubbed HMAFlow, to improve optical flow estimation in challenging scenes.
The proposed model mainly consists of two core components: a Hierarchical Motion Field Alignment (HMA) module and a Correlation Self-Attention (CSA) module.
Experimental results demonstrate that our model achieves the best generalization performance compared to other state-of-the-art methods.
arXiv Detail & Related papers (2024-09-09T11:43:35Z) - TIDE: Temporally Incremental Disparity Estimation via Pattern Flow in
Structured Light System [17.53719804060679]
TIDE-Net is a learning-based technique for disparity computation in mono-camera structured light systems.
We exploit the deformation of projected patterns (named pattern flow) on captured image sequences to model the temporal information.
For each incoming frame, our model fuses correlation volumes (from current frame) and disparity (from former frame) warped by pattern flow.
arXiv Detail & Related papers (2023-10-13T07:55:33Z) - GAFlow: Incorporating Gaussian Attention into Optical Flow [62.646389181507764]
We push Gaussian Attention (GA) into the optical flow models to accentuate local properties during representation learning.
We introduce a novel Gaussian-Constrained Layer (GCL) which can be easily plugged into existing Transformer blocks.
For reliable motion analysis, we provide a new Gaussian-Guided Attention Module (GGAM)
arXiv Detail & Related papers (2023-09-28T07:46:01Z) - Deep Dynamic Scene Deblurring from Optical Flow [53.625999196063574]
Deblurring can provide visually more pleasant pictures and make photography more convenient.
It is difficult to model the non-uniform blur mathematically.
We develop a convolutional neural network (CNN) to restore the sharp images from the deblurred features.
arXiv Detail & Related papers (2023-01-18T06:37:21Z) - RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos [28.995525297929348]
RealFlow is a framework that can create large-scale optical flow datasets directly from unlabeled realistic videos.
We first estimate optical flow between a pair of video frames, and then synthesize a new image from this pair based on the predicted flow.
Our approach achieves state-of-the-art performance on two standard benchmarks compared with both supervised and unsupervised optical flow methods.
arXiv Detail & Related papers (2022-07-22T13:33:03Z) - Deep Equilibrium Optical Flow Estimation [80.80992684796566]
Recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms.
These RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation.
We propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer.
arXiv Detail & Related papers (2022-04-18T17:53:44Z) - Learning Optical Flow from a Few Matches [67.83633948984954]
We show that the dense correlation volume representation is redundant and accurate flow estimation can be achieved with only a fraction of elements in it.
Experiments show that our method can reduce computational cost and memory use significantly, while maintaining high accuracy.
arXiv Detail & Related papers (2021-04-05T21:44:00Z) - DeFlow: Learning Complex Image Degradations from Unpaired Data with
Conditional Flows [145.83812019515818]
We propose DeFlow, a method for learning image degradations from unpaired data.
We model the degradation process in the latent space of a shared flow-decoder network.
We validate our DeFlow formulation on the task of joint image restoration and super-resolution.
arXiv Detail & Related papers (2021-01-14T18:58:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.