VLAD-BuFF: Burst-aware Fast Feature Aggregation for Visual Place Recognition

• URL: http://arxiv.org/abs/2409.19293v1
• Date: Sat, 28 Sep 2024 09:44:08 GMT
• Title: VLAD-BuFF: Burst-aware Fast Feature Aggregation for Visual Place Recognition
• Authors: Ahmad Khaliq, Ming Xu, Stephen Hausler, Michael Milford, Sourav Garg,
• Abstract summary: This paper introduces VLAD-BuFF, a self-similarity based feature discounting mechanism to learn Burst-aware features within end-to-end VPR training. We benchmark our method on 9 public datasets, where VLAD-BuFF sets a new state of the art. Our method is able to maintain its high recall even for 12x reduced local feature dimensions, thus enabling fast feature aggregation without compromising on recall.
• Score: 23.173085268845384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual Place Recognition (VPR) is a crucial component of many visual localization pipelines for embodied agents. VPR is often formulated as an image retrieval task aimed at jointly learning local features and an aggregation method. The current state-of-the-art VPR methods rely on VLAD aggregation, which can be trained to learn a weighted contribution of features through their soft assignment to cluster centers. However, this process has two key limitations. Firstly, the feature-to-cluster weighting does not account for over-represented repetitive structures within a cluster, e.g., shadows or window panes; this phenomenon is also referred to as the `burstiness' problem, classically solved by discounting repetitive features before aggregation. Secondly, feature to cluster comparisons are compute-intensive for state-of-the-art image encoders with high-dimensional local features. This paper addresses these limitations by introducing VLAD-BuFF with two novel contributions: i) a self-similarity based feature discounting mechanism to learn Burst-aware features within end-to-end VPR training, and ii) Fast Feature aggregation by reducing local feature dimensions specifically through PCA-initialized learnable pre-projection. We benchmark our method on 9 public datasets, where VLAD-BuFF sets a new state of the art. Our method is able to maintain its high recall even for 12x reduced local feature dimensions, thus enabling fast feature aggregation without compromising on recall. Through additional qualitative studies, we show how our proposed weighting method effectively downweights the non-distinctive features. Source code: https://github.com/Ahmedest61/VLAD-BuFF/.

Related papers

• PVAFN: Point-Voxel Attention Fusion Network with Multi-Pooling Enhancing for 3D Object Detection [59.355022416218624]
  integration of point and voxel representations is becoming more common in LiDAR-based 3D object detection. We propose a novel two-stage 3D object detector, called Point-Voxel Attention Fusion Network (PVAFN) PVAFN uses a multi-pooling strategy to integrate both multi-scale and region-specific information effectively.
  arXiv  Detail & Related papers  (2024-08-26T19:43:01Z)
• A Refreshed Similarity-based Upsampler for Direct High-Ratio Feature Upsampling [54.05517338122698]
  We propose an explicitly controllable query-key feature alignment from both semantic-aware and detail-aware perspectives. We also develop a fine-grained neighbor selection strategy on HR features, which is simple yet effective for alleviating mosaic artifacts. Our proposed ReSFU framework consistently achieves satisfactory performance on different segmentation applications.
  arXiv  Detail & Related papers  (2024-07-02T14:12:21Z)
• Local positional graphs and attentive local features for a data and runtime-efficient hierarchical place recognition pipeline [11.099588962062937]
  This paper proposes a runtime and data-efficient hierarchical VPR pipeline that extends existing approaches and presents novel ideas. First, we propose Local Positional Graphs (LPG), a training-free and runtime-efficient approach to encode spatial context information of local image features. Second, we present Attentive Local SPED (ATLAS), an extension of our previous local features approach with an attention module. Third, we present a hierarchical pipeline that exploits hyperdimensional computing to use the same local features as holistic HDC-descriptors for fast candidate selection and for candidate reranking.
  arXiv  Detail & Related papers  (2024-03-15T13:26:39Z)
• Deep Homography Estimation for Visual Place Recognition [49.235432979736395]
  We propose a transformer-based deep homography estimation (DHE) network. It takes the dense feature map extracted by a backbone network as input and fits homography for fast and learnable geometric verification. Experiments on benchmark datasets show that our method can outperform several state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-25T13:22:17Z)
• Optimal Transport Aggregation for Visual Place Recognition [9.192660643226372]
  We introduce SALAD, which reformulates NetVLAD's soft-assignment of local features to clusters as an optimal transport problem. In SALAD, we consider both feature-to-cluster and cluster-to-feature relations and we also introduce a 'dustbin' cluster, designed to selectively discard features deemed non-informative. Our single-stage method surpasses single-stage baselines in public VPR datasets, but also surpasses two-stage methods that add a re-ranking with significantly higher cost.
  arXiv  Detail & Related papers  (2023-11-27T15:46:19Z)
• Holistic Prototype Attention Network for Few-Shot VOS [74.25124421163542]
  Few-shot video object segmentation (FSVOS) aims to segment dynamic objects of unseen classes by resorting to a small set of support images. We propose a holistic prototype attention network (HPAN) for advancing FSVOS.
  arXiv  Detail & Related papers  (2023-07-16T03:48:57Z)
• PARFormer: Transformer-based Multi-Task Network for Pedestrian Attribute Recognition [23.814762073093153]
  We propose a pure transformer-based multi-task PAR network named PARFormer, which includes four modules. In the feature extraction module, we build a strong baseline for feature extraction, which achieves competitive results on several PAR benchmarks. In the viewpoint perception module, we explore the impact of viewpoints on pedestrian attributes, and propose a multi-view contrastive loss. In the attribute recognition module, we alleviate the negative-positive imbalance problem to generate the attribute predictions.
  arXiv  Detail & Related papers  (2023-04-14T16:27:56Z)
• Learning Implicit Feature Alignment Function for Semantic Segmentation [51.36809814890326]
  Implicit Feature Alignment function (IFA) is inspired by the rapidly expanding topic of implicit neural representations. We show that IFA implicitly aligns the feature maps at different levels and is capable of producing segmentation maps in arbitrary resolutions. Our method can be combined with improvement on various architectures, and it achieves state-of-the-art accuracy trade-off on common benchmarks.
  arXiv  Detail & Related papers  (2022-06-17T09:40:14Z)
• M2IOSR: Maximal Mutual Information Open Set Recognition [47.1393314282815]
  We propose a mutual information-based method with a streamlined architecture for open set recognition. The proposed method significantly improves the performance of baselines and achieves new state-of-the-art results on several benchmarks consistently.
  arXiv  Detail & Related papers  (2021-08-05T05:08:12Z)
• Channel DropBlock: An Improved Regularization Method for Fine-Grained Visual Classification [58.07257910065007]
  Existing approaches mainly tackle this problem by introducing attention mechanisms to locate the discriminative parts or feature encoding approaches to extract the highly parameterized features in a weakly-supervised fashion. In this work, we propose a lightweight yet effective regularization method named Channel DropBlock (CDB) in combination with two alternative correlation metrics, to address this problem.
  arXiv  Detail & Related papers  (2021-06-07T09:03:02Z)
• STA-VPR: Spatio-temporal Alignment for Visual Place Recognition [17.212503755962757]
  We propose an adaptive dynamic time warping algorithm to align local features from the spatial domain while measuring the distance between two images. A local matching DTW algorithm is applied to perform image sequence matching based on temporal alignment. The results show that the proposed method significantly improves the CNN-based methods.
  arXiv  Detail & Related papers  (2021-03-25T03:27:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.