Learning Feature Matching via Matchable Keypoint-Assisted Graph Neural Network

• URL: http://arxiv.org/abs/2307.01447v1
• Date: Tue, 4 Jul 2023 02:50:44 GMT
• Title: Learning Feature Matching via Matchable Keypoint-Assisted Graph Neural Network
• Authors: Zizhuo Li and Jiayi Ma
• Abstract summary: Accurately matching local features between a pair of images is a challenging computer vision task. Previous studies typically use attention based graph neural networks (GNNs) with fully-connected graphs over keypoints within/across images. We propose MaKeGNN, a sparse attention-based GNN architecture which bypasses non-repeatable keypoints and leverages matchable ones to guide message passing.
• Score: 52.29330138835208
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurately matching local features between a pair of images is a challenging computer vision task. Previous studies typically use attention based graph neural networks (GNNs) with fully-connected graphs over keypoints within/across images for visual and geometric information reasoning. However, in the context of feature matching, considerable keypoints are non-repeatable due to occlusion and failure of the detector, and thus irrelevant for message passing. The connectivity with non-repeatable keypoints not only introduces redundancy, resulting in limited efficiency, but also interferes with the representation aggregation process, leading to limited accuracy. Targeting towards high accuracy and efficiency, we propose MaKeGNN, a sparse attention-based GNN architecture which bypasses non-repeatable keypoints and leverages matchable ones to guide compact and meaningful message passing. More specifically, our Bilateral Context-Aware Sampling Module first dynamically samples two small sets of well-distributed keypoints with high matchability scores from the image pair. Then, our Matchable Keypoint-Assisted Context Aggregation Module regards sampled informative keypoints as message bottlenecks and thus constrains each keypoint only to retrieve favorable contextual information from intra- and inter- matchable keypoints, evading the interference of irrelevant and redundant connectivity with non-repeatable ones. Furthermore, considering the potential noise in initial keypoints and sampled matchable ones, the MKACA module adopts a matchability-guided attentional aggregation operation for purer data-dependent context propagation. By these means, we achieve the state-of-the-art performance on relative camera estimation, fundamental matrix estimation, and visual localization, while significantly reducing computational and memory complexity compared to typical attentional GNNs.

Related papers

• Utilizing Radiomic Feature Analysis For Automated MRI Keypoint Detection: Enhancing Graph Applications [2.8084568003406316]
  Graph neural networks (GNNs) present a promising alternative to CNNs and transformers in certain image processing applications. One approach involves converting images into nodes by identifying significant keypoints within them. This research sets the stage for expanding GNN applications into various applications, including but not limited to image classification, segmentation, and registration.
  arXiv  Detail & Related papers  (2023-11-30T06:37:02Z)
• ResMatch: Residual Attention Learning for Local Feature Matching [51.07496081296863]
  We rethink cross- and self-attention from the viewpoint of traditional feature matching and filtering. We inject the similarity of descriptors and relative positions into cross- and self-attention score. We mine intra- and inter-neighbors according to the similarity of descriptors and relative positions.
  arXiv  Detail & Related papers  (2023-07-11T11:32:12Z)
• FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network [48.912196729711624]
  Few-shot semantic segmentation is the task of learning to locate each pixel of a novel class in a query image with only a few annotated support images. We propose a Feature-Enhanced Context-Aware Network (FECANet) to suppress the matching noise caused by inter-class local similarity. In addition, we propose a novel correlation reconstruction module that encodes extra correspondence relations between foreground and background and multi-scale context semantic features.
  arXiv  Detail & Related papers  (2023-01-19T16:31:13Z)
• ClusTR: Exploring Efficient Self-attention via Clustering for Vision Transformers [70.76313507550684]
  We propose a content-based sparse attention method, as an alternative to dense self-attention. Specifically, we cluster and then aggregate key and value tokens, as a content-based method of reducing the total token count. The resulting clustered-token sequence retains the semantic diversity of the original signal, but can be processed at a lower computational cost.
  arXiv  Detail & Related papers  (2022-08-28T04:18:27Z)
• Rethinking Query-Key Pairwise Interactions in Vision Transformers [5.141895475956681]
  We propose key-only attention, which excludes query-key pairwise interactions and uses a compute-efficient saliency-gate to obtain attention weights. We develop a new self-attention model family, LinGlos, which reach state-of-the-art accuracies on the parameter-limited setting of ImageNet classification benchmark.
  arXiv  Detail & Related papers  (2022-07-01T03:36:49Z)
• ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching [15.620335576962475]
  ClusterGNN is an attentional GNN architecture which operates on clusters for learning the feature matching task. Our approach yields a 59.7% reduction in runtime and 58.4% reduction in memory consumption for dense detection.
  arXiv  Detail & Related papers  (2022-04-25T14:43:15Z)
• DenseGAP: Graph-Structured Dense Correspondence Learning with Anchor Points [15.953570826460869]
  Establishing dense correspondence between two images is a fundamental computer vision problem. We introduce DenseGAP, a new solution for efficient Dense correspondence learning with a Graph-structured neural network conditioned on Anchor Points. Our method advances the state-of-the-art of correspondence learning on most benchmarks.
  arXiv  Detail & Related papers  (2021-12-13T18:59:30Z)
• Probabilistic Spatial Distribution Prior Based Attentional Keypoints Matching Network [19.708243062836104]
  Keypoints matching is a pivotal component for many image-relevant applications such as image stitching, visual simultaneous localization and mapping. In this paper, we demonstrate that the motion estimation from IMU integration can be used to exploit the spatial distribution prior of keypoints between images. We present a projection loss for the proposed keypoints matching network, which gives a smooth edge between matching and un-matching keypoints.
  arXiv  Detail & Related papers  (2021-11-17T09:52:03Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)
• Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread [136.2224792151324]
  We propose a new deep learning algorithm for fast salient object detection. The proposed algorithm achieves competitive accuracy and high inference efficiency simultaneously with a single CPU thread.
  arXiv  Detail & Related papers  (2020-01-22T15:23:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.