Towards Transformer-based Homogenization of Satellite Imagery for Landsat-8 and Sentinel-2

• URL: http://arxiv.org/abs/2210.07654v1
• Date: Fri, 14 Oct 2022 09:13:34 GMT
• Title: Towards Transformer-based Homogenization of Satellite Imagery for Landsat-8 and Sentinel-2
• Authors: Venkatesh Thirugnana Sambandham, Konstantin Kirchheim, Sayan Mukhopadhaya, Frank Ortmeier
• Abstract summary: Landsat-8 (NASA) and Sentinel-2 (ESA) are two prominent multi-spectral imaging satellite projects that provide publicly available data. This work provides a first glance at the possibility of using a transformer-based model to reduce the spectral and spatial differences between observations from both satellite projects.
• Score: 1.4699455652461728
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Landsat-8 (NASA) and Sentinel-2 (ESA) are two prominent multi-spectral imaging satellite projects that provide publicly available data. The multi-spectral imaging sensors of the satellites capture images of the earth's surface in the visible and infrared region of the electromagnetic spectrum. Since the majority of the earth's surface is constantly covered with clouds, which are not transparent at these wavelengths, many images do not provide much information. To increase the temporal availability of cloud-free images of a certain area, one can combine the observations from multiple sources. However, the sensors of satellites might differ in their properties, making the images incompatible. This work provides a first glance at the possibility of using a transformer-based model to reduce the spectral and spatial differences between observations from both satellite projects. We compare the results to a model based on a fully convolutional UNet architecture. Somewhat surprisingly, we find that, while deep models outperform classical approaches, the UNet significantly outperforms the transformer in our experiments.

Related papers

• Deep Multimodal Fusion for Semantic Segmentation of Remote Sensing Earth Observation Data [0.08192907805418582]
  This paper proposes a late fusion deep learning model (LF-DLM) for semantic segmentation. One branch integrates detailed textures from aerial imagery captured by UNetFormer with a Multi-Axis Vision Transformer (ViT) backbone. The other branch captures complex-temporal dynamics from the Sentinel-2 satellite imageMax time series using a U-ViNet with Temporal Attention (U-TAE)
  arXiv  Detail & Related papers  (2024-10-01T07:50:37Z)
• Detection of Moving Objects in Earth Observation Satellite Images [0.0]
  We assess the feasibility of detecting moving objects and measuring their velocities in one particular archive of satellite images. Our results indicate that the movement of common transportation vehicles, airplanes, cars, and boats, can be detected and measured.
  arXiv  Detail & Related papers  (2024-05-18T20:55:49Z)
• SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
  We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks. To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
  arXiv  Detail & Related papers  (2024-03-25T10:30:22Z)
• DiffusionSat: A Generative Foundation Model for Satellite Imagery [63.2807119794691]
  We present DiffusionSat, to date the largest generative foundation model trained on a collection of publicly available large, high-resolution remote sensing datasets. Our method produces realistic samples and can be used to solve multiple generative tasks including temporal generation, superresolution given multi-spectral inputs and in-painting.
  arXiv  Detail & Related papers  (2023-12-06T16:53:17Z)
• Diffusion Models for Interferometric Satellite Aperture Radar [73.01013149014865]
  Probabilistic Diffusion Models (PDMs) have recently emerged as a very promising class of generative models. Here, we leverage PDMs to generate several radar-based satellite image datasets. We show that PDMs succeed in generating images with complex and realistic structures, but that sampling time remains an issue.
  arXiv  Detail & Related papers  (2023-08-31T16:26:17Z)
• Energy-Based Models for Cross-Modal Localization using Convolutional Transformers [52.27061799824835]
  We present a novel framework for localizing a ground vehicle mounted with a range sensor against satellite imagery in the absence of GPS. We propose a method using convolutional transformers that performs accurate metric-level localization in a cross-modal manner. We train our model end-to-end and demonstrate our approach achieving higher accuracy than the state-of-the-art on KITTI, Pandaset, and a custom dataset.
  arXiv  Detail & Related papers  (2023-06-06T21:27:08Z)
• Deep Learning Models for River Classification at Sub-Meter Resolutions from Multispectral and Panchromatic Commercial Satellite Imagery [2.121978045345352]
  This study focuses on rivers in the Arctic, using images from the Quickbird, WorldView, and GeoEye satellites. We use the RGB, and NIR bands of the 8-band multispectral sensors. Those trained models all achieve excellent precision and recall over 90% on validation data, aided by on-the-fly preprocessing of the training data specific to satellite imagery. In a novel approach, we then use results from the multispectral model to generate training data for FCN that only require panchromatic imagery, of which considerably more is available.
  arXiv  Detail & Related papers  (2022-12-27T20:56:34Z)
• Processing Images from Multiple IACTs in the TAIGA Experiment with Convolutional Neural Networks [62.997667081978825]
  We use convolutional neural networks (CNNs) to analyze Monte Carlo-simulated images from the TAIGA experiment. The analysis includes selection of the images corresponding to the showers caused by gamma rays and estimating the energy of the gamma rays.
  arXiv  Detail & Related papers  (2021-12-31T10:49:11Z)
• Big Plastic Masses Detection using Sentinel 2 Images [91.3755431537592]
  This communication describes a preliminary research on detection of big masses of plastic (marine litter) on the oceans and seas using EO (Earth Observation) satellite systems.
  arXiv  Detail & Related papers  (2021-03-17T10:45:33Z)
• Spectral Synthesis for Satellite-to-Satellite Translation [6.266622997342922]
  Earth observing satellites carrying multi-spectral sensors are widely used to monitor the physical and biological states of the atmosphere, land, and oceans. These satellites have different vantage points above the earth and different spectral imaging bands resulting in inconsistent imagery from one to another. We tackle the problem of generating synthetic spectral imagery for multispectral sensors as an unsupervised image-to-image translation problem with partial labels.
  arXiv  Detail & Related papers  (2020-10-12T21:36:39Z)
• Cross-Sensor Adversarial Domain Adaptation of Landsat-8 and Proba-V images for Cloud Detection [1.5828697880068703]
  The number of Earth observation satellites carrying optical sensors with similar characteristics is constantly growing. Differences in retrieved radiances lead to significant drops in accuracy, which hampers knowledge and information sharing across sensors. We propose a domain adaptation to reduce the statistical differences between images of two satellite sensors in order to boost the performance of transfer learning models.
  arXiv  Detail & Related papers  (2020-06-10T16:16:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.