Learning a Sensor-invariant Embedding of Satellite Data: A Case Study for Lake Ice Monitoring

• URL: http://arxiv.org/abs/2107.09092v1
• Date: Mon, 19 Jul 2021 18:11:55 GMT
• Title: Learning a Sensor-invariant Embedding of Satellite Data: A Case Study for Lake Ice Monitoring
• Authors: Manu Tom, Yuchang Jiang, Emmanuel Baltsavias, Konrad Schindler
• Abstract summary: We learn a joint, sensor-invariant embedding within a deep neural network. Our application problem is the monitoring of lake ice on Alpine lakes. By fusing satellite data, we map lake ice at a temporal resolution of 1.5 days.
• Score: 19.72060218456938
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fusing satellite imagery acquired with different sensors has been a long-standing challenge of Earth observation, particularly across different modalities such as optical and Synthetic Aperture Radar (SAR) images. Here, we explore the joint analysis of imagery from different sensors in the light of representation learning: we propose to learn a joint, sensor-invariant embedding (feature representation) within a deep neural network. Our application problem is the monitoring of lake ice on Alpine lakes. To reach the temporal resolution requirement of the Swiss Global Climate Observing System (GCOS) office, we combine three image sources: Sentinel-1 SAR (S1-SAR), Terra MODIS and Suomi-NPP VIIRS. The large gaps between the optical and SAR domains and between the sensor resolutions make this a challenging instance of the sensor fusion problem. Our approach can be classified as a feature-level fusion that is learnt in a data-driven manner. The proposed network architecture has separate encoding branches for each image sensor, which feed into a single latent embedding. I.e., a common feature representation shared by all inputs, such that subsequent processing steps deliver comparable output irrespective of which sort of input image was used. By fusing satellite data, we map lake ice at a temporal resolution of <1.5 days. The network produces spatially explicit lake ice maps with pixel-wise accuracies >91.3% (respectively, mIoU scores >60.7%) and generalises well across different lakes and winters. Moreover, it sets a new state-of-the-art for determining the important ice-on and ice-off dates for the target lakes, in many cases meeting the GCOS requirement.

Related papers

• Sea ice detection using concurrent multispectral and synthetic aperture radar imagery [1.0400484498567675]
  This paper proposes a new tool trained using concurrent multispectral Visible and SAR imagery for sea Ice Detection (ViSual_IceD) ViSual_IceD is a convolution neural network (CNN) that builds on the classic U-Net architecture by containing two parallel encoders. As the spatial-temporal coverage of MSI and SAR imagery continues to increase, ViSual_IceD provides a new opportunity for accurate sea ice coverage detection in polar regions.
  arXiv  Detail & Related papers  (2024-01-11T16:14:30Z)
• DAM-Net: Global Flood Detection from SAR Imagery Using Differential Attention Metric-Based Vision Transformers [22.885444177106873]
  Detection of flooded areas using high-resolution synthetic aperture radar (SAR) imagery is a critical task with applications in crisis and disaster management. To address this issue, we propose a novel differential attention metric-based network (DAM-Net) in this study. The DAM-Net comprises two key components: a weight-sharing Siamese backbone to obtain multi-scale change features of multi-temporal images and tokens containing high-level semantic information of water-body changes.
  arXiv  Detail & Related papers  (2023-06-01T14:12:33Z)
• DensePose From WiFi [86.61881052177228]
  We develop a deep neural network that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions. Our model can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches.
  arXiv  Detail & Related papers  (2022-12-31T16:48:43Z)
• Deep Learning Models for River Classification at Sub-Meter Resolutions from Multispectral and Panchromatic Commercial Satellite Imagery [2.121978045345352]
  This study focuses on rivers in the Arctic, using images from the Quickbird, WorldView, and GeoEye satellites. We use the RGB, and NIR bands of the 8-band multispectral sensors. Those trained models all achieve excellent precision and recall over 90% on validation data, aided by on-the-fly preprocessing of the training data specific to satellite imagery. In a novel approach, we then use results from the multispectral model to generate training data for FCN that only require panchromatic imagery, of which considerably more is available.
  arXiv  Detail & Related papers  (2022-12-27T20:56:34Z)
• xView3-SAR: Detecting Dark Fishing Activity Using Synthetic Aperture Radar Imagery [52.67592123500567]
  Unsustainable fishing practices worldwide pose a major threat to marine resources and ecosystems. It is now possible to automate detection of dark vessels day or night, under all-weather conditions. xView3-SAR consists of nearly 1,000 analysis-ready SAR images from the Sentinel-1 mission.
  arXiv  Detail & Related papers  (2022-06-02T06:53:45Z)
• Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU [58.720142291102135]
  In this paper we propose a pose estimation software exploiting neural network architectures. We show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space.
  arXiv  Detail & Related papers  (2022-04-07T08:53:18Z)
• Learning Enriched Illuminants for Cross and Single Sensor Color Constancy [182.4997117953705]
  We propose cross-sensor self-supervised training to train the network. We train the network by randomly sampling the artificial illuminants in a sensor-independent manner. Experiments show that our cross-sensor model and single-sensor model outperform other state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-03-21T15:45:35Z)
• The QXS-SAROPT Dataset for Deep Learning in SAR-Optical Data Fusion [14.45289690639374]
  We publish the QXS-SAROPT dataset to foster deep learning research in SAR-optical data fusion. We show exemplary results for two representative applications, namely SAR-optical image matching and SAR ship detection boosted by cross-modal information from optical images.
  arXiv  Detail & Related papers  (2021-03-15T10:22:46Z)
• Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images [193.77450545067967]
  We propose an end-to-end Dense Attention Fluid Network (DAFNet) for salient object detection in optical remote sensing images (RSIs) A Global Context-aware Attention (GCA) module is proposed to adaptively capture long-range semantic context relationships. We construct a new and challenging optical RSI dataset for SOD that contains 2,000 images with pixel-wise saliency annotations.
  arXiv  Detail & Related papers  (2020-11-26T06:14:10Z)
• Ice Monitoring in Swiss Lakes from Optical Satellites and Webcams using Machine Learning [16.71718177904522]
  Multi-temporal satellite images publicly available webcam images are among a viable data sources to monitor lake ice. In this work we investigate machine learning-based image analysis as a tool to determine the extent of ice on Swiss Alpine lakes. We model lake ice monitoring as a pixel-wise semantic segmentation problem, i.e., each pixel on the lake surface is classified to obtain a spatially explicit map of ice cover.
  arXiv  Detail & Related papers  (2020-10-27T14:02:00Z)
• Depth Estimation from Monocular Images and Sparse Radar Data [93.70524512061318]
  In this paper, we explore the possibility of achieving a more accurate depth estimation by fusing monocular images and Radar points using a deep neural network. We find that the noise existing in Radar measurements is one of the main key reasons that prevents one from applying the existing fusion methods. The experiments are conducted on the nuScenes dataset, which is one of the first datasets which features Camera, Radar, and LiDAR recordings in diverse scenes and weather conditions.
  arXiv  Detail & Related papers  (2020-09-30T19:01:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.