Unifying Local and Global Multimodal Features for Place Recognition in Aliased and Low-Texture Environments

• URL: http://arxiv.org/abs/2403.13395v1
• Date: Wed, 20 Mar 2024 08:35:57 GMT
• Title: Unifying Local and Global Multimodal Features for Place Recognition in Aliased and Low-Texture Environments
• Authors: Alberto García-Hernández, Riccardo Giubilato, Klaus H. Strobl, Javier Civera, Rudolph Triebel,
• Abstract summary: This paper presents a novel model, called UMF, that leverages multi-modality by cross-attention blocks between vision and LiDAR features. Experiments, particularly on sequences captured on a planetary-analogous environment, show that UMF outperforms significantly previous baselines. Our work aims to enhance the reliability of SLAM in all situations, we also explore its performance on the widely used RobotCar dataset.
• Score: 19.859565090638167
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Perceptual aliasing and weak textures pose significant challenges to the task of place recognition, hindering the performance of Simultaneous Localization and Mapping (SLAM) systems. This paper presents a novel model, called UMF (standing for Unifying Local and Global Multimodal Features) that 1) leverages multi-modality by cross-attention blocks between vision and LiDAR features, and 2) includes a re-ranking stage that re-orders based on local feature matching the top-k candidates retrieved using a global representation. Our experiments, particularly on sequences captured on a planetary-analogous environment, show that UMF outperforms significantly previous baselines in those challenging aliased environments. Since our work aims to enhance the reliability of SLAM in all situations, we also explore its performance on the widely used RobotCar dataset, for broader applicability. Code and models are available at https://github.com/DLR-RM/UMF

Related papers

• INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model [71.50973774576431]
  We propose a novel MLLM, INF-LLaVA, designed for effective high-resolution image perception. We introduce a Dual-perspective Cropping Module (DCM), which ensures that each sub-image contains continuous details from a local perspective. Second, we introduce Dual-perspective Enhancement Module (DEM) to enable the mutual enhancement of global and local features.
  arXiv  Detail & Related papers  (2024-07-23T06:02:30Z)
• Other Tokens Matter: Exploring Global and Local Features of Vision Transformers for Object Re-Identification [63.147482497821166]
  We first explore the influence of global and local features of ViT and then propose a novel Global-Local Transformer (GLTrans) for high-performance object Re-ID. Our proposed method achieves superior performance on four object Re-ID benchmarks.
  arXiv  Detail & Related papers  (2024-04-23T12:42:07Z)
• Modality-Collaborative Transformer with Hybrid Feature Reconstruction for Robust Emotion Recognition [35.15390769958969]
  We propose a unified framework, Modality-Collaborative Transformer with Hybrid Feature Reconstruction (MCT-HFR) MCT-HFR consists of a novel attention-based encoder which concurrently extracts and dynamically balances the intra- and inter-modality relations. During model training, LFI leverages complete features as supervisory signals to recover local missing features, while GFA is designed to reduce the global semantic gap between pairwise complete and incomplete representations.
  arXiv  Detail & Related papers  (2023-12-26T01:59:23Z)
• Optimization Efficient Open-World Visual Region Recognition [55.76437190434433]
  RegionSpot integrates position-aware localization knowledge from a localization foundation model with semantic information from a ViL model. Experiments in open-world object recognition show that our RegionSpot achieves significant performance gain over prior alternatives.
  arXiv  Detail & Related papers  (2023-11-02T16:31:49Z)
• FS-Real: Towards Real-World Cross-Device Federated Learning [60.91678132132229]
  Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data. There is still a considerable gap between the flourishing FL research and real-world scenarios, mainly caused by the characteristics of heterogeneous devices and its scales. We propose an efficient and scalable prototyping system for real-world cross-device FL, FS-Real.
  arXiv  Detail & Related papers  (2023-03-23T15:37:17Z)
• Style-Hallucinated Dual Consistency Learning: A Unified Framework for Visual Domain Generalization [113.03189252044773]
  We propose a unified framework, Style-HAllucinated Dual consistEncy learning (SHADE), to handle domain shift in various visual tasks. Our versatile SHADE can significantly enhance the generalization in various visual recognition tasks, including image classification, semantic segmentation and object detection.
  arXiv  Detail & Related papers  (2022-12-18T11:42:51Z)
• RING++: Roto-translation Invariant Gram for Global Localization on a Sparse Scan Map [20.276334172402763]
  We propose RING++ which has roto-translation invariant representation for place recognition, and global convergence for both rotation and translation estimation. With the theoretical guarantee, RING++ is able to address the large viewpoint difference using a lightweight map with sparse scans. This is the first learning-free framework to address all subtasks of global localization in the sparse scan map.
  arXiv  Detail & Related papers  (2022-10-12T07:49:24Z)
• Cross-modal Local Shortest Path and Global Enhancement for Visible-Thermal Person Re-Identification [2.294635424666456]
  We propose the Cross-modal Local Shortest Path and Global Enhancement (CM-LSP-GE) modules,a two-stream network based on joint learning of local and global features. The experimental results on two typical datasets show that our model is obviously superior to the most state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-09T10:27:22Z)
• Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition [29.282413482297255]
  This paper introduces Patch-NetVLAD, which provides a novel formulation for combining the advantages of both local and global descriptor methods. We show that Patch-NetVLAD outperforms both global and local feature descriptor-based methods with comparable compute. It is also adaptable to user requirements, with a speed-optimised version operating over an order of magnitude faster than the state-of-the-art.
  arXiv  Detail & Related papers  (2021-03-02T05:53:32Z)
• Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation [28.721376937882958]
  Gait recognition is one of the most important biometric technologies and has been applied in many fields. Recent gait recognition frameworks represent each gait frame by descriptors extracted from either global appearances or local regions of humans. We propose a novel feature extraction and fusion framework to achieve discriminative feature representations for gait recognition.
  arXiv  Detail & Related papers  (2020-11-03T04:07:13Z)
• Global Context-Aware Progressive Aggregation Network for Salient Object Detection [117.943116761278]
  We propose a novel network named GCPANet to integrate low-level appearance features, high-level semantic features, and global context features. We show that the proposed approach outperforms the state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-03-02T04:26:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.