Cross-view Geo-localization with Evolving Transformer

• URL: http://arxiv.org/abs/2107.00842v2
• Date: Mon, 5 Jul 2021 02:23:48 GMT
• Title: Cross-view Geo-localization with Evolving Transformer
• Authors: Hongji Yang, Xiufan Lu and Yingying Zhu
• Abstract summary: Cross-view geo-localization is challenging due to drastic appearance and geometry differences across views. We devise a novel geo-localization Transformer (EgoTR) that utilizes the properties of self-attention in Transformer to model global dependencies. Our EgoTR performs favorably against state-of-the-art methods on standard, fine-grained and cross-dataset cross-view geo-localization tasks.
• Score: 7.5800316275498645
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this work, we address the problem of cross-view geo-localization, which estimates the geospatial location of a street view image by matching it with a database of geo-tagged aerial images. The cross-view matching task is extremely challenging due to drastic appearance and geometry differences across views. Unlike existing methods that predominantly fall back on CNN, here we devise a novel evolving geo-localization Transformer (EgoTR) that utilizes the properties of self-attention in Transformer to model global dependencies, thus significantly decreasing visual ambiguities in cross-view geo-localization. We also exploit the positional encoding of Transformer to help the EgoTR understand and correspond geometric configurations between ground and aerial images. Compared to state-of-the-art methods that impose strong assumption on geometry knowledge, the EgoTR flexibly learns the positional embeddings through the training objective and hence becomes more practical in many real-world scenarios. Although Transformer is well suited to our task, its vanilla self-attention mechanism independently interacts within image patches in each layer, which overlooks correlations between layers. Instead, this paper propose a simple yet effective self-cross attention mechanism to improve the quality of learned representations. The self-cross attention models global dependencies between adjacent layers, which relates between image patches while modeling how features evolve in the previous layer. As a result, the proposed self-cross attention leads to more stable training, improves the generalization ability and encourages representations to keep evolving as the network goes deeper. Extensive experiments demonstrate that our EgoTR performs favorably against state-of-the-art methods on standard, fine-grained and cross-dataset cross-view geo-localization tasks.

Related papers

• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• IPT-V2: Efficient Image Processing Transformer using Hierarchical Attentions [26.09373405194564]
  We present an efficient image processing transformer architecture with hierarchical attentions, called IPTV2. We adopt a focal context self-attention (FCSA) and a global grid self-attention (GGSA) to obtain adequate token interactions in local and global receptive fields. Our proposed IPT-V2 achieves state-of-the-art results on various image processing tasks, covering denoising, deblurring, deraining and obtains much better trade-off for performance and computational complexity than previous methods.
  arXiv  Detail & Related papers  (2024-03-31T10:01:20Z)
• ConGeo: Robust Cross-view Geo-localization across Ground View Variations [34.192775134189965]
  Cross-view geo-localization aims at localizing a ground-level query image by matching it to its corresponding geo-referenced aerial view. Existing learning pipelines are orientation-specific or FoV-specific, demanding separate model training for different ground view variations. We propose ConGeo, a single- and cross-view Contrastive method for Geo-localization.
  arXiv  Detail & Related papers  (2024-03-20T20:37:13Z)
• Cross-domain and Cross-dimension Learning for Image-to-Graph Transformers [50.576354045312115]
  Direct image-to-graph transformation is a challenging task that solves object detection and relationship prediction in a single model. We introduce a set of methods enabling cross-domain and cross-dimension transfer learning for image-to-graph transformers. We demonstrate our method's utility in cross-domain and cross-dimension experiments, where we pretrain our models on 2D satellite images before applying them to vastly different target domains in 2D and 3D.
  arXiv  Detail & Related papers  (2024-03-11T10:48:56Z)
• Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection [76.11864242047074]
  We propose a novel Affine-Consistent Transformer (AC-Former), which directly yields a sequence of nucleus positions. We introduce an Adaptive Affine Transformer (AAT) module, which can automatically learn the key spatial transformations to warp original images for local network training. Experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art algorithms on various benchmarks.
  arXiv  Detail & Related papers  (2023-10-22T02:27:02Z)
• Cross-view Geo-localization via Learning Disentangled Geometric Layout Correspondence [11.823147814005411]
  Cross-view geo-localization aims to estimate the location of a query ground image by matching it to a reference geo-tagged aerial images database. Recent works achieve outstanding progress on cross-view geo-localization benchmarks. However, existing methods still suffer from poor performance on the cross-area benchmarks.
  arXiv  Detail & Related papers  (2022-12-08T04:54:01Z)
• TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization [81.70547404891099]
  CNN-based methods for cross-view image geo-localization fail to model global correlation. We propose a pure transformer-based approach (TransGeo) to address these limitations. TransGeo achieves state-of-the-art results on both urban and rural datasets.
  arXiv  Detail & Related papers  (2022-03-31T21:19:41Z)
• Co-visual pattern augmented generative transformer learning for automobile geo-localization [12.449657263683337]
  Cross-view geo-localization (CVGL) aims to estimate the geographical location of the ground-level camera by matching against enormous geo-tagged aerial images. We present a novel approach using cross-view knowledge generative techniques in combination with transformers, namely mutual generative transformer learning (MGTL) for CVGL.
  arXiv  Detail & Related papers  (2022-03-17T07:29:02Z)
• Improving the generalization of network based relative pose regression: dimension reduction as a regularizer [16.63174637692875]
  State-of-the-art visual localization methods perform pose estimation using geometry based solver within the RANSAC framework. End-to-end learning based regression networks provide a solution to circumvent the requirement for precise pixel-level correspondences. In this paper, we explicitly add a learnable matching layer within the network to isolate the pose regression solver from the absolute image feature values. We implement this dimension regularization strategy within a two-layer pyramid based framework to regress the localization results from coarse to fine.
  arXiv  Detail & Related papers  (2020-10-24T06:20:46Z)
• Multi-Level Graph Convolutional Network with Automatic Graph Learning for Hyperspectral Image Classification [63.56018768401328]
  We propose a Multi-level Graph Convolutional Network (GCN) with Automatic Graph Learning method (MGCN-AGL) for HSI classification. By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions. Our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level.
  arXiv  Detail & Related papers  (2020-09-19T09:26:20Z)
• Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching [95.64702426906466]
  Cross-view geo-localization is a problem given a large-scale database of geo-tagged aerial images. Knowing orientation between ground and aerial images can significantly reduce matching ambiguity between these two views. We design a Dynamic Similarity Matching network to estimate cross-view orientation alignment during localization.
  arXiv  Detail & Related papers  (2020-05-08T05:21:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.