Searching from Area to Point: A Hierarchical Framework for Semantic-Geometric Combined Feature Matching

• URL: http://arxiv.org/abs/2305.00194v5
• Date: Thu, 2 May 2024 03:19:33 GMT
• Title: Searching from Area to Point: A Hierarchical Framework for Semantic-Geometric Combined Feature Matching
• Authors: Yesheng Zhang, Xu Zhao, Dahong Qian,
• Abstract summary: We set the initial search space for point matching as the matched image areas containing prominent semantic, named semantic area matches. This search space favors point matching by salient features and alleviates the accuracy limitation in recent Transformer-based matching methods. We propose a hierarchical feature matching framework: Area to Point Matching (A2PM), to first find semantic area matches between images and later perform point matching on area matches.
• Score: 15.781946287115561
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Feature matching is a crucial technique in computer vision. A unified perspective for this task is to treat it as a searching problem, aiming at an efficient search strategy to narrow the search space to point matches between images. One of the key aspects of search strategy is the search space, which in current approaches is not carefully defined, resulting in limited matching accuracy. This paper, thus, pays attention to the search space and proposes to set the initial search space for point matching as the matched image areas containing prominent semantic, named semantic area matches. This search space favors point matching by salient features and alleviates the accuracy limitation in recent Transformer-based matching methods. To achieve this search space, we introduce a hierarchical feature matching framework: Area to Point Matching (A2PM), to first find semantic area matches between images and later perform point matching on area matches. We further propose Semantic and Geometry Area Matching (SGAM) method to realize this framework, which utilizes semantic prior and geometry consistency to establish accurate area matches between images. By integrating SGAM with off-the-shelf state-of-the-art matchers, our method, adopting the A2PM framework, achieves encouraging precision improvements in massive point matching and pose estimation experiments.

Related papers

• A Feature Matching Method Based on Multi-Level Refinement Strategy [11.300618381337777]
  Experimental results demonstrate that the KTGP-ORB method reduces the error by an average of 29.92% compared to the ORB algorithm in complex scenes with illumination variations and blur.
  arXiv  Detail & Related papers  (2024-02-21T02:57:27Z)
• MESA: Matching Everything by Segmenting Anything [16.16319526547664]
  MESA is a novel approach to establish precise area (or region) matches for efficient matching redundancy reduction. We show that MESA yields substantial precision improvement for multiple point matchers in indoor and outdoor downstream tasks.
  arXiv  Detail & Related papers  (2024-01-30T04:39:32Z)
• Learning-based Relational Object Matching Across Views [63.63338392484501]
  We propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network.
  arXiv  Detail & Related papers  (2023-05-03T19:36:51Z)
• Location-Aware Self-Supervised Transformers [74.76585889813207]
  We propose to pretrain networks for semantic segmentation by predicting the relative location of image parts. We control the difficulty of the task by masking a subset of the reference patch features visible to those of the query. Our experiments show that this location-aware pretraining leads to representations that transfer competitively to several challenging semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T16:24:29Z)
• A Geometrically Constrained Point Matching based on View-invariant Cross-ratios, and Homography [2.050924050557755]
  A geometrically constrained algorithm is proposed to verify the correctness of initially matched SIFT keypoints based on view-invariant cross-ratios (CRs) By randomly forming pentagons from these keypoints and matching their shape and location among images with CRs, robust planar region estimation can be achieved efficiently. Experimental results show that satisfactory results can be obtained for various scenes with single as well as multiple planar regions.
  arXiv  Detail & Related papers  (2022-11-06T01:55:35Z)
• ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer [33.603064903549985]
  ASpanFormer is a Transformer-based detector-free matcher that is built on hierarchical attention structure. We propose a novel attention operation which is capable of adjusting attention span in a self-adaptive manner. By these means, we are able to not only maintain long-range dependencies, but also enable fine-grained attention among pixels of high relevance.
  arXiv  Detail & Related papers  (2022-08-30T12:21:15Z)
• Dense Siamese Network [86.23741104851383]
  We present Dense Siamese Network (DenseSiam), a simple unsupervised learning framework for dense prediction tasks. It learns visual representations by maximizing the similarity between two views of one image with two types of consistency, i.e., pixel consistency and region consistency. It surpasses state-of-the-art segmentation methods by 2.1 mIoU with 28% training costs.
  arXiv  Detail & Related papers  (2022-03-21T15:55:23Z)
• Exploring Complicated Search Spaces with Interleaving-Free Sampling [127.07551427957362]
  In this paper, we build the search algorithm upon a complicated search space with long-distance connections. We present a simple yet effective algorithm named textbfIF-NAS, where we perform a periodic sampling strategy to construct different sub-networks. In the proposed search space, IF-NAS outperform both random sampling and previous weight-sharing search algorithms by a significant margin.
  arXiv  Detail & Related papers  (2021-12-05T06:42:48Z)
• SIFT Matching by Context Exposed [7.99536002595393]
  This paper investigates how to step up local image descriptor matching by exploiting matching context information. A new matching strategy and a novel local spatial filter, named respectively blob matching and Delaunay Triangulation Matching (DTM) are devised. DTM is comparable or better than the state-of-the-art in terms of matching accuracy and robustness, especially for non-planar scenes.
  arXiv  Detail & Related papers  (2021-06-17T15:10:59Z)
• Affinity Space Adaptation for Semantic Segmentation Across Domains [57.31113934195595]
  In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation. Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains. We develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment.
  arXiv  Detail & Related papers  (2020-09-26T10:28:11Z)
• Making Affine Correspondences Work in Camera Geometry Computation [62.7633180470428]
  Local features provide region-to-region rather than point-to-point correspondences. We propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. Experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times.
  arXiv  Detail & Related papers  (2020-07-20T12:07:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.