Learning Probabilistic Coordinate Fields for Robust Correspondences

• URL: http://arxiv.org/abs/2306.04231v1
• Date: Wed, 7 Jun 2023 08:14:17 GMT
• Title: Learning Probabilistic Coordinate Fields for Robust Correspondences
• Authors: Weiyue Zhao, Hao Lu, Xinyi Ye, Zhiguo Cao, Xin Li
• Abstract summary: We introduce Probabilistic Coordinate Fields (PCFs), a novel geometric-invariant coordinate representation for image correspondence problems. We implement PCFs in a probabilistic network termed PCF-Net, which parameterizes the distribution of coordinate fields as Gaussian mixture models. Experiments on both indoor and outdoor datasets suggest that accurate geometric invariant coordinates help to achieve the state of the art in several correspondence problems.
• Score: 24.01047640087573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Probabilistic Coordinate Fields (PCFs), a novel geometric-invariant coordinate representation for image correspondence problems. In contrast to standard Cartesian coordinates, PCFs encode coordinates in correspondence-specific barycentric coordinate systems (BCS) with affine invariance. To know \textit{when and where to trust} the encoded coordinates, we implement PCFs in a probabilistic network termed PCF-Net, which parameterizes the distribution of coordinate fields as Gaussian mixture models. By jointly optimizing coordinate fields and their confidence conditioned on dense flows, PCF-Net can work with various feature descriptors when quantifying the reliability of PCFs by confidence maps. An interesting observation of this work is that the learned confidence map converges to geometrically coherent and semantically consistent regions, which facilitates robust coordinate representation. By delivering the confident coordinates to keypoint/feature descriptors, we show that PCF-Net can be used as a plug-in to existing correspondence-dependent approaches. Extensive experiments on both indoor and outdoor datasets suggest that accurate geometric invariant coordinates help to achieve the state of the art in several correspondence problems, such as sparse feature matching, dense image registration, camera pose estimation, and consistency filtering. Further, the interpretable confidence map predicted by PCF-Net can also be leveraged to other novel applications from texture transfer to multi-homography classification.

Related papers

• Geometry-aware Feature Matching for Large-Scale Structure from Motion [10.645087195983201]
  We introduce geometry cues in addition to color cues to fill gaps when there is less overlap in large-scale scenarios. Our approach ensures that the denser correspondences from detector-free methods are geometrically consistent and more accurate. It outperforms state-of-the-art feature matching methods on benchmark datasets.
  arXiv  Detail & Related papers  (2024-09-03T21:41:35Z)
• Thinner Latent Spaces: Detecting dimension and imposing invariance through autoencoder gradient constraints [9.380902608139902]
  We show that orthogonality relations within the latent layer of the network can be leveraged to infer the intrinsic dimensionality of nonlinear manifold data sets. We outline the relevant theory relying on differential geometry, and describe the corresponding gradient-descent optimization algorithm.
  arXiv  Detail & Related papers  (2024-08-28T20:56:35Z)
• A2B: Anchor to Barycentric Coordinate for Robust Correspondence [25.719939636977934]
  We show that geometric-invariant coordinate representations, such as barycentric coordinates, can significantly reduce mismatches between features. We introduce DEGREE, a novel anchor-to-barycentric (A2B) coordinate encoding approach, which generates multiple affine-invariant correspondence coordinates from paired images.
  arXiv  Detail & Related papers  (2023-06-05T10:28:53Z)
• Learning Implicit Feature Alignment Function for Semantic Segmentation [51.36809814890326]
  Implicit Feature Alignment function (IFA) is inspired by the rapidly expanding topic of implicit neural representations. We show that IFA implicitly aligns the feature maps at different levels and is capable of producing segmentation maps in arbitrary resolutions. Our method can be combined with improvement on various architectures, and it achieves state-of-the-art accuracy trade-off on common benchmarks.
  arXiv  Detail & Related papers  (2022-06-17T09:40:14Z)
• Trading Positional Complexity vs. Deepness in Coordinate Networks [33.90893096003318]
  We show that alternative non-Fourier embedding functions can indeed be used for positional encoding. Their performance is entirely determined by a trade-off between the stable rank of the embedded matrix and the distance preservation between embedded coordinates. We argue that employing a more complex positional encoding -- that scales exponentially with the number of modes -- requires only a linear (rather than deep) coordinate function to achieve comparable performance.
  arXiv  Detail & Related papers  (2022-05-18T15:17:09Z)
• DenseGAP: Graph-Structured Dense Correspondence Learning with Anchor Points [15.953570826460869]
  Establishing dense correspondence between two images is a fundamental computer vision problem. We introduce DenseGAP, a new solution for efficient Dense correspondence learning with a Graph-structured neural network conditioned on Anchor Points. Our method advances the state-of-the-art of correspondence learning on most benchmarks.
  arXiv  Detail & Related papers  (2021-12-13T18:59:30Z)
• PDC-Net+: Enhanced Probabilistic Dense Correspondence Network [161.76275845530964]
  Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, capable of estimating accurate dense correspondences. We develop an architecture and an enhanced training strategy tailored for robust and generalizable uncertainty prediction. Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets.
  arXiv  Detail & Related papers  (2021-09-28T17:56:41Z)
• Similarity-Aware Fusion Network for 3D Semantic Segmentation [87.51314162700315]
  We propose a similarity-aware fusion network (SAFNet) to adaptively fuse 2D images and 3D point clouds for 3D semantic segmentation. We employ a late fusion strategy where we first learn the geometric and contextual similarities between the input and back-projected (from 2D pixels) point clouds. We show that SAFNet significantly outperforms existing state-of-the-art fusion-based approaches across various data integrity.
  arXiv  Detail & Related papers  (2021-07-04T09:28:18Z)
• A Point-Cloud Deep Learning Framework for Prediction of Fluid Flow Fields on Irregular Geometries [62.28265459308354]
  Network learns end-to-end mapping between spatial positions and CFD quantities. Incompress laminar steady flow past a cylinder with various shapes for its cross section is considered. Network predicts the flow fields hundreds of times faster than our conventional CFD.
  arXiv  Detail & Related papers  (2020-10-15T12:15:02Z)
• 3D Correspondence Grouping with Compatibility Features [51.869670613445685]
  We present a simple yet effective method for 3D correspondence grouping. The objective is to accurately classify initial correspondences obtained by matching local geometric descriptors into inliers and outliers. We propose a novel representation for 3D correspondences, dubbed compatibility feature (CF), to describe the consistencies within inliers and inconsistencies within outliers.
  arXiv  Detail & Related papers  (2020-07-21T02:39:48Z)
• Making Affine Correspondences Work in Camera Geometry Computation [62.7633180470428]
  Local features provide region-to-region rather than point-to-point correspondences. We propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. Experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times.
  arXiv  Detail & Related papers  (2020-07-20T12:07:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.