Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations

• URL: http://arxiv.org/abs/2203.04445v1
• Date: Tue, 8 Mar 2022 23:32:33 GMT
• Title: Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations
• Authors: Sachith Seneviratne, Kerry A. Nice, Jasper S. Wijnands, Mark Stevenson, Jason Thompson
• Abstract summary: We show that self-supervised methods can build a generalizable representation from as few as 200 cities. We also find that the performance discrepancy of such methods, when compared to supervised methods, is significant for remote sensing imagery.
• Score: 7.860343491689477
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervision based deep learning classification approaches have received considerable attention in academic literature. However, the performance of such methods on remote sensing imagery domains remains under-explored. In this work, we explore contrastive representation learning methods on the task of imagery-based city classification, an important problem in urban computing. We use satellite and map imagery across 2 domains, 3 million locations and more than 1500 cities. We show that self-supervised methods can build a generalizable representation from as few as 200 cities, with representations achieving over 95\% accuracy in unseen cities with minimal additional training. We also find that the performance discrepancy of such methods, when compared to supervised methods, induced by the domain discrepancy between natural imagery and abstract imagery is significant for remote sensing imagery. We compare all analysis against existing supervised models from academic literature and open-source our models for broader usage and further criticism.

Related papers

• Detecting Generated Images by Real Images Only [64.12501227493765]
  Existing generated image detection methods detect visual artifacts in generated images or learn discriminative features from both real and generated images by massive training. This paper approaches the generated image detection problem from a new perspective: Start from real images. By finding the commonality of real images and mapping them to a dense subspace in feature space, the goal is that generated images, regardless of their generative model, are then projected outside the subspace.
  arXiv  Detail & Related papers  (2023-11-02T03:09:37Z)
• Saliency Guided Contrastive Learning on Scene Images [71.07412958621052]
  We leverage the saliency map derived from the model's output during learning to highlight discriminative regions and guide the whole contrastive learning. Our method significantly improves the performance of self-supervised learning on scene images by +1.1, +4.3, +2.2 Top1 accuracy in ImageNet linear evaluation, Semi-supervised learning with 1% and 10% ImageNet labels, respectively.
  arXiv  Detail & Related papers  (2023-02-22T15:54:07Z)
• Mitigating Urban-Rural Disparities in Contrastive Representation Learning with Satellite Imagery [19.93324644519412]
  We consider the risk of urban-rural disparities in identification of land-cover features. We propose fair dense representation with contrastive learning (FairDCL) as a method for de-biasing the multi-level latent space of convolution neural network models. The obtained image representation mitigates downstream urban-rural prediction disparities and outperforms state-of-the-art baselines on real-world satellite images.
  arXiv  Detail & Related papers  (2022-11-16T04:59:46Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• Urban feature analysis from aerial remote sensing imagery using self-supervised and semi-supervised computer vision [8.124947412639704]
  Analysis of overhead imagery using computer vision is a problem that has received considerable attention in academic literature. These problems are addressed here through the development of a more generic framework, incorporating advances in representation learning. The successful low-level detection of urban infrastructure evolution over a 10-year period from 60 million unlabeled images, exemplifies the substantial potential of our approach to advance quantitative urban research.
  arXiv  Detail & Related papers  (2022-08-17T03:41:56Z)
• Segmenting across places: The need for fair transfer learning with satellite imagery [24.087993065704527]
  State-of-the-art models have better overall accuracy in rural areas compared to urban areas. We show that raw satellite images are overall more dissimilar between source and target districts for rural than for urban locations.
  arXiv  Detail & Related papers  (2022-04-09T02:14:56Z)
• Semi-Supervised Contrastive Learning for Remote Sensing: Identifying Ancient Urbanization in the South Central Andes [11.489739686646647]
  In this study, we use 95,358 unlabeled images and 5,830 labelled images in order to solve the issues associated with detecting ancient buildings from a long-tailed satellite image dataset. Our semi-supervised contrastive learning model achieved a promising testing balanced accuracy of 79.0%, which is a 3.8% improvement as compared to other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-12-13T06:26:47Z)
• Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities [81.29441139530844]
  This paper provides a systematic survey of deep learning methods for remote sensing image scene classification by covering more than 160 papers. We discuss the main challenges of remote sensing image scene classification and survey. We introduce the benchmarks used for remote sensing image scene classification and summarize the performance of more than two dozen representative algorithms.
  arXiv  Detail & Related papers  (2020-05-03T14:18:00Z)
• Extending and Analyzing Self-Supervised Learning Across Domains [50.13326427158233]
  Self-supervised representation learning has achieved impressive results in recent years. Experiments primarily come on ImageNet or other similarly large internet imagery datasets. We experiment with several popular methods on an unprecedented variety of domains.
  arXiv  Detail & Related papers  (2020-04-24T21:18:02Z)
• Geometrically Mappable Image Features [85.81073893916414]
  Vision-based localization of an agent in a map is an important problem in robotics and computer vision. We propose a method that learns image features targeted for image-retrieval-based localization.
  arXiv  Detail & Related papers  (2020-03-21T15:36:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.