Land Use Prediction using Electro-Optical to SAR Few-Shot Transfer Learning

• URL: http://arxiv.org/abs/2212.03084v1
• Date: Sun, 4 Dec 2022 22:41:25 GMT
• Title: Land Use Prediction using Electro-Optical to SAR Few-Shot Transfer Learning
• Authors: Marcel Hussing, Karen Li, Eric Eaton
• Abstract summary: Deep learning methods can facilitate the analysis of different satellite modalities, such as electro-optical (EO) and synthetic aperture radar (SAR) imagery. We show how distributional alignment of neural network embeddings can produce powerful transfer learning models by employing a sliced Wasserstein distance (SWD) loss. In an application to few-shot Local Climate Zone (LCZ) prediction, we show that these networks outperform multiple common baselines on datasets with a large number of classes.
• Score: 16.71560262537924
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Satellite image analysis has important implications for land use, urbanization, and ecosystem monitoring. Deep learning methods can facilitate the analysis of different satellite modalities, such as electro-optical (EO) and synthetic aperture radar (SAR) imagery, by supporting knowledge transfer between the modalities to compensate for individual shortcomings. Recent progress has shown how distributional alignment of neural network embeddings can produce powerful transfer learning models by employing a sliced Wasserstein distance (SWD) loss. We analyze how this method can be applied to Sentinel-1 and -2 satellite imagery and develop several extensions toward making it effective in practice. In an application to few-shot Local Climate Zone (LCZ) prediction, we show that these networks outperform multiple common baselines on datasets with a large number of classes. Further, we provide evidence that instance normalization can significantly stabilize the training process and that explicitly shaping the embedding space using supervised contrastive learning can lead to improved performance.

Related papers

• SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
  We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks. To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
  arXiv  Detail & Related papers  (2024-03-25T10:30:22Z)
• Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery [78.43828998065071]
  Recent advances in unsupervised learning have demonstrated the ability of large vision models to achieve promising results on downstream tasks. Such pre-training techniques have also been explored recently in the remote sensing domain due to the availability of large amount of unlabelled data. In this paper, we re-visit transformers pre-training and leverage multi-scale information that is effectively utilized with multiple modalities.
  arXiv  Detail & Related papers  (2024-03-08T16:18:04Z)
• Super-Resolution of BVOC Emission Maps Via Domain Adaptation [17.819699053848197]
  SuperResolution (SR) methods based on Deep Learning (DL) have been proposed, leveraging data from numerical simulations for their training process. In our work, we aim at super-resolving low resolution emission maps derived from satellite observations by leveraging the information of emission maps obtained through numerical simulations. We investigate the effectiveness of Domain Adaptation (DA) strategies at different stages by systematically varying the number of simulated and observed emissions used.
  arXiv  Detail & Related papers  (2023-06-22T10:59:15Z)
• Learning Sentinel-2 reflectance dynamics for data-driven assimilation and forecasting [11.0735899248545]
  We train a deep learning model inspired from the Koopman operator theory to model long-term reflectance dynamics in an unsupervised way. We show that this trained model, being differentiable, can be used as a prior for data assimilation in a straightforward way.
  arXiv  Detail & Related papers  (2023-05-05T10:04:03Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Evaluating the Label Efficiency of Contrastive Self-Supervised Learning for Multi-Resolution Satellite Imagery [0.0]
  Self-supervised learning has been applied in the remote sensing domain to exploit readily-available unlabeled data. In this paper, we study self-supervised visual representation learning through the lens of label efficiency.
  arXiv  Detail & Related papers  (2022-10-13T06:54:13Z)
• Semantic Segmentation of Vegetation in Remote Sensing Imagery Using Deep Learning [77.34726150561087]
  We propose an approach for creating a multi-modal and large-temporal dataset comprised of publicly available Remote Sensing data. We use Convolutional Neural Networks (CNN) models that are capable of separating different classes of vegetation.
  arXiv  Detail & Related papers  (2022-09-28T18:51:59Z)
• TRAIL: Near-Optimal Imitation Learning with Suboptimal Data [100.83688818427915]
  We present training objectives that use offline datasets to learn a factored transition model. Our theoretical analysis shows that the learned latent action space can boost the sample-efficiency of downstream imitation learning. To learn the latent action space in practice, we propose TRAIL (Transition-Reparametrized Actions for Imitation Learning), an algorithm that learns an energy-based transition model.
  arXiv  Detail & Related papers  (2021-10-27T21:05:00Z)
• Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection [50.29565896287595]
  We apply transfer learning to exploit common datasets for sarcasm detection. We propose a generalized latent optimization strategy that allows different losses to accommodate each other. In particular, we achieve 10.02% absolute performance gain over the previous state of the art on the iSarcasm dataset.
  arXiv  Detail & Related papers  (2021-04-19T13:07:52Z)
• Visualization of Deep Transfer Learning In SAR Imagery [0.0]
  We consider transfer learning to leverage deep features from a network trained on an EO ships dataset. By exploring the network activations in the form of class-activation maps, we gain insight on how a deep network interprets a new modality.
  arXiv  Detail & Related papers  (2021-03-20T00:16:15Z)
• Weakly-supervised land classification for coastal zone based on deep convolutional neural networks by incorporating dual-polarimetric characteristics into training dataset [1.125851164829582]
  We explore the performance of DCNNs on semantic segmentation using spaceborne polarimetric synthetic aperture radar (PolSAR) datasets. The semantic segmentation task using PolSAR data can be categorized as weakly supervised learning when the characteristics of SAR data and data annotating procedures are factored in. Three DCNN models, including SegNet, U-Net, and LinkNet, are implemented next.
  arXiv  Detail & Related papers  (2020-03-30T17:32:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.