Temporal Attention for Cross-View Sequential Image Localization

• URL: http://arxiv.org/abs/2408.15569v1
• Date: Wed, 28 Aug 2024 06:53:08 GMT
• Title: Temporal Attention for Cross-View Sequential Image Localization
• Authors: Dong Yuan, Frederic Maire, Feras Dayoub,
• Abstract summary: This paper introduces a novel approach to enhancing cross-view localization, focusing on the fine-grained, sequential localization of street-view images within a single known satellite image patch. By expanding to sequential image fine-grained localization, our model, equipped with a novel Temporal Attention Module (TAM), leverages contextual information to significantly improve sequential image localization accuracy.
• Score: 17.14320442129364
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces a novel approach to enhancing cross-view localization, focusing on the fine-grained, sequential localization of street-view images within a single known satellite image patch, a significant departure from traditional one-to-one image retrieval methods. By expanding to sequential image fine-grained localization, our model, equipped with a novel Temporal Attention Module (TAM), leverages contextual information to significantly improve sequential image localization accuracy. Our method shows substantial reductions in both mean and median localization errors on the Cross-View Image Sequence (CVIS) dataset, outperforming current state-of-the-art single-image localization techniques. Additionally, by adapting the KITTI-CVL dataset into sequential image sets, we not only offer a more realistic dataset for future research but also demonstrate our model's robust generalization capabilities across varying times and areas, evidenced by a 75.3% reduction in mean distance error in cross-view sequential image localization.

Related papers

• AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization [57.34659640776723]
  We propose an end-to-end framework named AddressCLIP to solve the problem with more semantics. We have built three datasets from Pittsburgh and San Francisco on different scales specifically for the IAL problem.
  arXiv  Detail & Related papers  (2024-07-11T03:18:53Z)
• Dual-Image Enhanced CLIP for Zero-Shot Anomaly Detection [58.228940066769596]
  We introduce a Dual-Image Enhanced CLIP approach, leveraging a joint vision-language scoring system. Our methods process pairs of images, utilizing each as a visual reference for the other, thereby enriching the inference process with visual context. Our approach significantly exploits the potential of vision-language joint anomaly detection and demonstrates comparable performance with current SOTA methods across various datasets.
  arXiv  Detail & Related papers  (2024-05-08T03:13:20Z)
• LIP-Loc: LiDAR Image Pretraining for Cross-Modal Localization [0.9562145896371785]
  We apply Contrastive Language-Image Pre-Training to the domains of 2D image and 3D LiDAR points on the task of cross-modal localization. Our method outperforms state-of-the-art recall@1 accuracy on the KITTI-360 dataset by 22.4%, using only perspective images. We also demonstrate the zero-shot capabilities of our model and we beat SOTA by 8% without even training on it.
  arXiv  Detail & Related papers  (2023-12-27T17:23:57Z)
• LIME: Localized Image Editing via Attention Regularization in Diffusion Models [74.3811832586391]
  This paper introduces LIME for localized image editing in diffusion models that do not require user-specified regions of interest (RoI) or additional text input. Our method employs features from pre-trained methods and a simple clustering technique to obtain precise semantic segmentation maps. We propose a novel cross-attention regularization technique that penalizes unrelated cross-attention scores in the RoI during the denoising steps, ensuring localized edits.
  arXiv  Detail & Related papers  (2023-12-14T18:59:59Z)
• CutPaste: Self-Supervised Learning for Anomaly Detection and Localization [59.719925639875036]
  We propose a framework for building anomaly detectors using normal training data only. We first learn self-supervised deep representations and then build a generative one-class classifier on learned representations. Our empirical study on MVTec anomaly detection dataset demonstrates the proposed algorithm is general to be able to detect various types of real-world defects.
  arXiv  Detail & Related papers  (2021-04-08T19:04:55Z)
• Spatially Consistent Representation Learning [12.120041613482558]
  We propose a spatially consistent representation learning algorithm (SCRL) for multi-object and location-specific tasks. We devise a novel self-supervised objective that tries to produce coherent spatial representations of a randomly cropped local region. On various downstream localization tasks with benchmark datasets, the proposed SCRL shows significant performance improvements.
  arXiv  Detail & Related papers  (2021-03-10T15:23:45Z)
• Cross-Descriptor Visual Localization and Mapping [81.16435356103133]
  Visual localization and mapping is the key technology underlying the majority of Mixed Reality and robotics systems. We present three novel scenarios for localization and mapping which require the continuous update of feature representations. Our data-driven approach is agnostic to the feature descriptor type, has low computational requirements, and scales linearly with the number of description algorithms.
  arXiv  Detail & Related papers  (2020-12-02T18:19:51Z)
• LM-Reloc: Levenberg-Marquardt Based Direct Visual Relocalization [54.77498358487812]
  LM-Reloc is a novel approach for visual relocalization based on direct image alignment. We propose a loss formulation inspired by the classical Levenberg-Marquardt algorithm to train LM-Net.
  arXiv  Detail & Related papers  (2020-10-13T12:15:20Z)
• Domain-invariant Similarity Activation Map Contrastive Learning for Retrieval-based Long-term Visual Localization [30.203072945001136]
  In this work, a general architecture is first formulated probabilistically to extract domain invariant feature through multi-domain image translation. And then a novel gradient-weighted similarity activation mapping loss (Grad-SAM) is incorporated for finer localization with high accuracy. Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMUSeasons dataset. Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision.
  arXiv  Detail & Related papers  (2020-09-16T14:43:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.