Guide Local Feature Matching by Overlap Estimation

• URL: http://arxiv.org/abs/2202.09050v1
• Date: Fri, 18 Feb 2022 07:11:36 GMT
• Title: Guide Local Feature Matching by Overlap Estimation
• Authors: Ying Chen, Dihe Huang, Shang Xu, Jianlin Liu, Yong Liu
• Abstract summary: We introduce a novel Overlap Estimation method conditioned on image pairs with TRansformer, named OETR. OETR performs overlap estimation in a two-step process of feature correlation and then overlap regression. Experiments show that OETR can boost state-of-the-art local feature matching performance substantially.
• Score: 9.387323456222823
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Local image feature matching under large appearance, viewpoint, and distance changes is challenging yet important. Conventional methods detect and match tentative local features across the whole images, with heuristic consistency checks to guarantee reliable matches. In this paper, we introduce a novel Overlap Estimation method conditioned on image pairs with TRansformer, named OETR, to constrain local feature matching in the commonly visible region. OETR performs overlap estimation in a two-step process of feature correlation and then overlap regression. As a preprocessing module, OETR can be plugged into any existing local feature detection and matching pipeline, to mitigate potential view angle or scale variance. Intensive experiments show that OETR can boost state-of-the-art local feature matching performance substantially, especially for image pairs with small shared regions. The code will be publicly available at https://github.com/AbyssGaze/OETR.

Related papers

• Leveraging Semantic Cues from Foundation Vision Models for Enhanced Local Feature Correspondence [12.602194710071116]
  This paper presents a new method that uses semantic cues from foundation vision model features to enhance local feature matching. We present adapted versions of six existing descriptors, with an average increase in performance of 29% in camera localization.
  arXiv  Detail & Related papers  (2024-10-12T13:45:26Z)
• Breaking the Frame: Visual Place Recognition by Overlap Prediction [53.17564423756082]
  We propose a novel visual place recognition approach based on overlap prediction, called VOP. VOP proceeds co-visible image sections by obtaining patch-level embeddings using a Vision Transformer backbone. Our approach uses a voting mechanism to assess overlap scores for potential database images.
  arXiv  Detail & Related papers  (2024-06-23T20:00:20Z)
• Deep Homography Estimation for Visual Place Recognition [49.235432979736395]
  We propose a transformer-based deep homography estimation (DHE) network. It takes the dense feature map extracted by a backbone network as input and fits homography for fast and learnable geometric verification. Experiments on benchmark datasets show that our method can outperform several state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-25T13:22:17Z)
• Adaptive Spot-Guided Transformer for Consistent Local Feature Matching [64.30749838423922]
  We propose Adaptive Spot-Guided Transformer (ASTR) for local feature matching. ASTR models the local consistency and scale variations in a unified coarse-to-fine architecture.
  arXiv  Detail & Related papers  (2023-03-29T12:28:01Z)
• Improving Transformer-based Image Matching by Cascaded Capturing Spatially Informative Keypoints [44.90917854990362]
  We propose a transformer-based cascade matching model -- Cascade feature Matching TRansformer (CasMTR) We use a simple yet effective Non-Maximum Suppression (NMS) post-process to filter keypoints through the confidence map. CasMTR achieves state-of-the-art performance in indoor and outdoor pose estimation as well as visual localization.
  arXiv  Detail & Related papers  (2023-03-06T04:32:34Z)
• OAMatcher: An Overlapping Areas-based Network for Accurate Local Feature Matching [9.006654114778073]
  We propose OAMatcher, a detector-free method that imitates humans behavior to generate dense and accurate matches. OAMatcher predicts overlapping areas to promote effective and clean global context aggregation. Comprehensive experiments demonstrate that OAMatcher outperforms the state-of-the-art methods on several benchmarks.
  arXiv  Detail & Related papers  (2023-02-12T03:32:45Z)
• Towards Effective Image Manipulation Detection with Proposal Contrastive Learning [61.5469708038966]
  We propose Proposal Contrastive Learning (PCL) for effective image manipulation detection. Our PCL consists of a two-stream architecture by extracting two types of global features from RGB and noise views respectively. Our PCL can be easily adapted to unlabeled data in practice, which can reduce manual labeling costs and promote more generalizable features.
  arXiv  Detail & Related papers  (2022-10-16T13:30:13Z)
• Local Relation Learning for Face Forgery Detection [73.73130683091154]
  We propose a novel perspective of face forgery detection via local relation learning. Specifically, we propose a Multi-scale Patch Similarity Module (MPSM), which measures the similarity between features of local regions. We also propose an RGB-Frequency Attention Module (RFAM) to fuse information in both RGB and frequency domains for more comprehensive local feature representation.
  arXiv  Detail & Related papers  (2021-05-06T10:44:32Z)
• LoFTR: Detector-Free Local Feature Matching with Transformers [40.754990768677295]
  Instead of performing image feature detection, description, and matching sequentially, we propose to first establish pixel-wise dense matches at a coarse level. In contrast to dense methods that use a cost volume to search correspondences, we use self and cross attention layers in Transformer to obtain feature descriptors that are conditioned on both images. The experiments on indoor and outdoor datasets show that LoFTR outperforms state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2021-04-01T17:59:42Z)
• Patch2Pix: Epipolar-Guided Pixel-Level Correspondences [38.38520763114715]
  We present Patch2Pix, a novel refinement network that refines match proposals by regressing pixel-level matches from the local regions defined by those proposals. We show that our refinement network significantly improves the performance of correspondence networks on image matching, homography estimation, and localization tasks.
  arXiv  Detail & Related papers  (2020-12-03T13:44:02Z)
• Making Affine Correspondences Work in Camera Geometry Computation [62.7633180470428]
  Local features provide region-to-region rather than point-to-point correspondences. We propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. Experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times.
  arXiv  Detail & Related papers  (2020-07-20T12:07:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.