Motion Estimation for Large Displacements and Deformations

• URL: http://arxiv.org/abs/2206.12464v1
• Date: Fri, 24 Jun 2022 18:53:22 GMT
• Title: Motion Estimation for Large Displacements and Deformations
• Authors: Qiao Chen, Charalambos Poullis
• Abstract summary: Variational optical flow techniques based on a coarse-to-fine scheme interpolate sparse matches and locally optimize an energy model conditioned on colour, gradient and smoothness. This paper addresses this problem and presents HybridFlow, a variational motion estimation framework for large displacements and deformations.
• Score: 7.99536002595393
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large displacement optical flow is an integral part of many computer vision tasks. Variational optical flow techniques based on a coarse-to-fine scheme interpolate sparse matches and locally optimize an energy model conditioned on colour, gradient and smoothness, making them sensitive to noise in the sparse matches, deformations, and arbitrarily large displacements. This paper addresses this problem and presents HybridFlow, a variational motion estimation framework for large displacements and deformations. A multi-scale hybrid matching approach is performed on the image pairs. Coarse-scale clusters formed by classifying pixels according to their feature descriptors are matched using the clusters' context descriptors. We apply a multi-scale graph matching on the finer-scale superpixels contained within each matched pair of coarse-scale clusters. Small clusters that cannot be further subdivided are matched using localized feature matching. Together, these initial matches form the flow, which is propagated by an edge-preserving interpolation and variational refinement. Our approach does not require training and is robust to substantial displacements and rigid and non-rigid transformations due to motion in the scene, making it ideal for large-scale imagery such as Wide-Area Motion Imagery (WAMI). More notably, HybridFlow works on directed graphs of arbitrary topology representing perceptual groups, which improves motion estimation in the presence of significant deformations. We demonstrate HybridFlow's superior performance to state-of-the-art variational techniques on two benchmark datasets and report comparable results with state-of-the-art deep-learning-based techniques.

Related papers

• Oscillation Inversion: Understand the structure of Large Flow Model through the Lens of Inversion Method [60.88467353578118]
  We show that a fixed-point-inspired iterative approach to invert real-world images does not achieve convergence, instead oscillating between distinct clusters. We introduce a simple and fast distribution transfer technique that facilitates image enhancement, stroke-based recoloring, as well as visual prompt-guided image editing.
  arXiv  Detail & Related papers  (2024-11-17T17:45:37Z)
• Spatially-Attentive Patch-Hierarchical Network with Adaptive Sampling for Motion Deblurring [34.751361664891235]
  We propose a pixel adaptive and feature attentive design for handling large blur variations across different spatial locations. We show that our approach performs favorably against the state-of-the-art deblurring algorithms.
  arXiv  Detail & Related papers  (2024-02-09T01:00:09Z)
• Graph Invariant Learning with Subgraph Co-mixup for Out-Of-Distribution Generalization [51.913685334368104]
  We propose a novel graph invariant learning method based on invariant and variant patterns co-mixup strategy. Our method significantly outperforms state-of-the-art under various distribution shifts.
  arXiv  Detail & Related papers  (2023-12-18T07:26:56Z)
• RGM: A Robust Generalizable Matching Model [49.60975442871967]
  We propose a deep model for sparse and dense matching, termed RGM (Robust Generalist Matching) To narrow the gap between synthetic training samples and real-world scenarios, we build a new, large-scale dataset with sparse correspondence ground truth. We are able to mix up various dense and sparse matching datasets, significantly improving the training diversity.
  arXiv  Detail & Related papers  (2023-10-18T07:30:08Z)
• Dynamic Kernel-Based Adaptive Spatial Aggregation for Learned Image Compression [63.56922682378755]
  We focus on extending spatial aggregation capability and propose a dynamic kernel-based transform coding. The proposed adaptive aggregation generates kernel offsets to capture valid information in the content-conditioned range to help transform. Experimental results demonstrate that our method achieves superior rate-distortion performance on three benchmarks compared to the state-of-the-art learning-based methods.
  arXiv  Detail & Related papers  (2023-08-17T01:34:51Z)
• Adaptive Single Image Deblurring [43.02281823557039]
  We propose an efficient pixel adaptive and feature attentive design for handling large blur variations within and across different images. We also propose an effective content-aware global-local filtering module that significantly improves the performance.
  arXiv  Detail & Related papers  (2022-01-01T10:10:19Z)
• Convolutional Hough Matching Networks for Robust and Efficient Visual Correspondence [41.061667361696465]
  We introduce a Hough transform perspective on convolutional matching and propose an effective geometric matching algorithm, dubbed Convolutional Hough Matching (CHM) Our method sets a new state of the art on standard benchmarks for semantic visual correspondence, proving its strong robustness to challenging intra-class variations.
  arXiv  Detail & Related papers  (2021-09-11T08:39:41Z)
• ResNet-LDDMM: Advancing the LDDMM Framework Using Deep Residual Networks [86.37110868126548]
  In this work, we make use of deep residual neural networks to solve the non-stationary ODE (flow equation) based on a Euler's discretization scheme. We illustrate these ideas on diverse registration problems of 3D shapes under complex topology-preserving transformations.
  arXiv  Detail & Related papers  (2021-02-16T04:07:13Z)
• A Flexible Framework for Designing Trainable Priors with Adaptive Smoothing and Game Encoding [57.1077544780653]
  We introduce a general framework for designing and training neural network layers whose forward passes can be interpreted as solving non-smooth convex optimization problems. We focus on convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions. This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end.
  arXiv  Detail & Related papers  (2020-06-26T08:34:54Z)
• Deep Transformation-Invariant Clustering [24.23117820167443]
  We present an approach that does not rely on abstract features but instead learns to predict image transformations. This learning process naturally fits in the gradient-based training of K-means and Gaussian mixture model. We demonstrate that our novel approach yields competitive and highly promising results on standard image clustering benchmarks.
  arXiv  Detail & Related papers  (2020-06-19T13:43:08Z)
• Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring [39.92889091819711]
  We propose an efficient pixel adaptive and feature attentive design for handling large blur variations across different spatial locations. We use a patch-hierarchical attentive architecture composed of the above module that implicitly discovers the spatial variations in the blur present in the input image. Our design offers significant improvements over the state-of-the-art in accuracy as well as speed.
  arXiv  Detail & Related papers  (2020-04-11T09:24:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.