Learning Second-Order Attentive Context for Efficient Correspondence Pruning

• URL: http://arxiv.org/abs/2303.15761v1
• Date: Tue, 28 Mar 2023 06:40:11 GMT
• Title: Learning Second-Order Attentive Context for Efficient Correspondence Pruning
• Authors: Xinyi Ye, Weiyue Zhao, Hao Lu, Zhiguo Cao
• Abstract summary: Correspondence pruning aims to search consistent correspondences (inliers) from a set of putative correspondences. In this paper, we propose an effective and efficient method for correspondence pruning.
• Score: 22.100653202605965
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Correspondence pruning aims to search consistent correspondences (inliers) from a set of putative correspondences. It is challenging because of the disorganized spatial distribution of numerous outliers, especially when putative correspondences are largely dominated by outliers. It's more challenging to ensure effectiveness while maintaining efficiency. In this paper, we propose an effective and efficient method for correspondence pruning. Inspired by the success of attentive context in correspondence problems, we first extend the attentive context to the first-order attentive context and then introduce the idea of attention in attention (ANA) to model second-order attentive context for correspondence pruning. Compared with first-order attention that focuses on feature-consistent context, second-order attention dedicates to attention weights itself and provides an additional source to encode consistent context from the attention map. For efficiency, we derive two approximate formulations for the naive implementation of second-order attention to optimize the cubic complexity to linear complexity, such that second-order attention can be used with negligible computational overheads. We further implement our formulations in a second-order context layer and then incorporate the layer in an ANA block. Extensive experiments demonstrate that our method is effective and efficient in pruning outliers, especially in high-outlier-ratio cases. Compared with the state-of-the-art correspondence pruning approach LMCNet, our method runs 14 times faster while maintaining a competitive accuracy.

Related papers

• Recycled Attention: Efficient inference for long-context language models [54.00118604124301]
  We propose Recycled Attention, an inference-time method which alternates between full context attention and attention over a subset of input tokens. When performing partial attention, we recycle the attention pattern of a previous token that has performed full attention and attend only to the top K most attended tokens. Compared to previously proposed inference-time acceleration method which attends only to local context or tokens with high accumulative attention scores, our approach flexibly chooses tokens that are relevant to the current decoding step.
  arXiv  Detail & Related papers  (2024-11-08T18:57:07Z)
• Fusion Makes Perfection: An Efficient Multi-Grained Matching Approach for Zero-Shot Relation Extraction [9.657656957845987]
  We propose an efficient multi-grained matching approach that uses virtual entity matching to reduce manual annotation cost. Experimental results show that our approach outperforms the previous State Of The Art (SOTA) methods.
  arXiv  Detail & Related papers  (2024-06-17T11:31:48Z)
• Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence [51.54175067684008]
  This paper introduces a Transformer-based integrative feature and cost aggregation network designed for dense matching tasks. We first show that feature aggregation and cost aggregation exhibit distinct characteristics and reveal the potential for substantial benefits stemming from the judicious use of both aggregation processes. Our framework is evaluated on standard benchmarks for semantic matching, and also applied to geometric matching, where we show that our approach achieves significant improvements compared to existing methods.
  arXiv  Detail & Related papers  (2024-03-17T07:02:55Z)
• Efficient Semantic Matching with Hypercolumn Correlation [58.92933923647451]
  HCCNet is an efficient yet effective semantic matching method. It exploits the full potential of multi-scale correlation maps. It eschews the reliance on expensive match-wise relationship mining on the 4D correlation map.
  arXiv  Detail & Related papers  (2023-11-07T20:40:07Z)
• ECO-TR: Efficient Correspondences Finding Via Coarse-to-Fine Refinement [80.94378602238432]
  We propose an efficient structure named Correspondence Efficient Transformer (ECO-TR) by finding correspondences in a coarse-to-fine manner. To achieve this, multiple transformer blocks are stage-wisely connected to gradually refine the predicted coordinates. Experiments on various sparse and dense matching tasks demonstrate the superiority of our method in both efficiency and effectiveness against existing state-of-the-arts.
  arXiv  Detail & Related papers  (2022-09-25T13:05:33Z)
• Towards Joint Intent Detection and Slot Filling via Higher-order Attention [47.78365472691051]
  Intent detection (ID) and Slot filling (SF) are two major tasks in spoken language understanding (SLU) We propose a Bilinear attention block, which exploits both the contextual and channel-wise bilinear attention distributions. We show that our approach yields improvements compared with the state-of-the-art approach.
  arXiv  Detail & Related papers  (2021-09-18T09:50:23Z)
• Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walk [59.87525177207915]
  We develop a self-supervised method to establish correspondences between two point clouds to approximate scene flow. Our method achieves state-of-the-art performance among self-supervised learning methods.
  arXiv  Detail & Related papers  (2021-05-18T03:12:42Z)
• ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting [108.93803186429017]
  End-to-end text-spotting aims to integrate detection and recognition in a unified framework. Here, we tackle end-to-end text spotting by presenting Adaptive Bezier Curve Network v2 (ABCNet v2) Our main contributions are four-fold: 1) For the first time, we adaptively fit arbitrarily-shaped text by a parameterized Bezier curve, which, compared with segmentation-based methods, can not only provide structured output but also controllable representation. Comprehensive experiments conducted on various bilingual (English and Chinese) benchmark datasets demonstrate that ABCNet v2 can achieve state-of-the
  arXiv  Detail & Related papers  (2021-05-08T07:46:55Z)
• S2DNet: Learning Accurate Correspondences for Sparse-to-Dense Feature Matching [36.48376198922595]
  S2DNet is a novel feature matching pipeline designed and trained to efficiently establish robust and accurate correspondences. We show that S2DNet achieves state-of-the-art results on the HPatches benchmark, as well as on several long-term visual localization datasets.
  arXiv  Detail & Related papers  (2020-04-03T17:04:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.