Visual Geo-localization with Self-supervised Representation Learning

• URL: http://arxiv.org/abs/2308.00090v2
• Date: Fri, 15 Sep 2023 20:48:12 GMT
• Title: Visual Geo-localization with Self-supervised Representation Learning
• Authors: Jiuhong Xiao, Gao Zhu and Giuseppe Loianno
• Abstract summary: We present a novel unified VG-SSL framework with the goal to enhance performance and training efficiency on a large Visual Geo-localization dataset. Our work incorporates multiple SSL methods tailored for VG: SimCLR, MoCov2, BYOL, SimSiam, Barlow Twins, and VICReg.
• Score: 8.642591824865892
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual Geo-localization (VG) has emerged as a significant research area, aiming to identify geolocation based on visual features. Most VG approaches use learnable feature extractors for representation learning. Recently, Self-Supervised Learning (SSL) methods have also demonstrated comparable performance to supervised methods by using numerous unlabeled images for representation learning. In this work, we present a novel unified VG-SSL framework with the goal to enhance performance and training efficiency on a large VG dataset by SSL methods. Our work incorporates multiple SSL methods tailored for VG: SimCLR, MoCov2, BYOL, SimSiam, Barlow Twins, and VICReg. We systematically analyze the performance of different training strategies and study the optimal parameter settings for the adaptation of SSL methods for the VG task. The results demonstrate that our method, without the significant computation and memory usage associated with Hard Negative Mining (HNM), can match or even surpass the VG performance of the baseline that employs HNM. The code is available at https://github.com/arplaboratory/VG_SSL.

Related papers

• A Closer Look at Benchmarking Self-Supervised Pre-training with Image Classification [51.35500308126506]
  Self-supervised learning (SSL) is a machine learning approach where the data itself provides supervision, eliminating the need for external labels. We study how classification-based evaluation protocols for SSL correlate and how well they predict downstream performance on different dataset types.
  arXiv  Detail & Related papers  (2024-07-16T23:17:36Z)
• GSLB: The Graph Structure Learning Benchmark [34.859275408785614]
  Graph Structure Learning (GSL) has recently garnered considerable attention due to its ability to optimize both the parameters of Graph Neural Networks (GNNs) and the computation graph structure simultaneously. There is no standard experimental setting or fair comparison for performance evaluation, which creates a great obstacle to understanding the progress in this field. We develop a comprehensive Graph Structure Learning Benchmark (GSLB) curated from 20 diverse graph datasets and 16 distinct GSL algorithms.
  arXiv  Detail & Related papers  (2023-10-08T14:13:03Z)
• MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments [72.6405488990753]
  Self-supervised learning can be used for mitigating the greedy needs of Vision Transformer networks. We propose a single-stage and standalone method, MOCA, which unifies both desired properties. We achieve new state-of-the-art results on low-shot settings and strong experimental results in various evaluation protocols.
  arXiv  Detail & Related papers  (2023-07-18T15:46:20Z)
• Simultaneous or Sequential Training? How Speech Representations Cooperate in a Multi-Task Self-Supervised Learning System [12.704529528199064]
  Recent work combined self-supervised learning (SSL) and visually grounded speech (VGS) processing mechanisms for representation learning. We study the joint optimization of wav2vec 2.0-based SSL and transformer-based VGS as a multi-task learning system.
  arXiv  Detail & Related papers  (2023-06-05T15:35:19Z)
• On the Transferability of Visual Features in Generalized Zero-Shot Learning [28.120004119724577]
  Generalized Zero-Shot Learning (GZSL) aims to train a classifier that can generalize to unseen classes. In this work, we investigate the utility of different GZSL methods when using different feature extractors. We also examine how these models' pre-training objectives, datasets, and architecture design affect their feature representation ability.
  arXiv  Detail & Related papers  (2022-11-22T18:59:09Z)
• Improving Self-Supervised Learning by Characterizing Idealized Representations [155.1457170539049]
  We prove necessary and sufficient conditions for any task invariant to given data augmentations. For contrastive learning, our framework prescribes simple but significant improvements to previous methods. For non-contrastive learning, we use our framework to derive a simple and novel objective.
  arXiv  Detail & Related papers  (2022-09-13T18:01:03Z)
• Industry Scale Semi-Supervised Learning for Natural Language Understanding [14.844450283047234]
  This paper presents a production Semi-Supervised Learning (SSL) pipeline based on the student-teacher framework. We investigate two questions related to the use of unlabeled data in production SSL context. We compare four widely used SSL techniques, Pseudo-Label (PL), Knowledge Distillation (KD), Virtual Adversarial Training (VAT) and Cross-View Training (CVT)
  arXiv  Detail & Related papers  (2021-03-29T18:24:02Z)
• Graph-based Semi-supervised Learning: A Comprehensive Review [51.26862262550445]
  Semi-supervised learning (SSL) has tremendous value in practice due to its ability to utilize both labeled data and unlabelled data. An important class of SSL methods is to naturally represent data as graphs, which corresponds to graph-based semi-supervised learning (GSSL) methods. GSSL methods have demonstrated their advantages in various domains due to their uniqueness of structure, the universality of applications, and their scalability to large scale data.
  arXiv  Detail & Related papers  (2021-02-26T05:11:09Z)
• Self-Supervised Learning of Graph Neural Networks: A Unified Review [50.71341657322391]
  Self-supervised learning is emerging as a new paradigm for making use of large amounts of unlabeled samples. We provide a unified review of different ways of training graph neural networks (GNNs) using SSL. Our treatment of SSL methods for GNNs sheds light on the similarities and differences of various methods, setting the stage for developing new methods and algorithms.
  arXiv  Detail & Related papers  (2021-02-22T03:43:45Z)
• On Data-Augmentation and Consistency-Based Semi-Supervised Learning [77.57285768500225]
  Recently proposed consistency-based Semi-Supervised Learning (SSL) methods have advanced the state of the art in several SSL tasks. Despite these advances, the understanding of these methods is still relatively limited.
  arXiv  Detail & Related papers  (2021-01-18T10:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.