UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Series

• URL: http://arxiv.org/abs/2304.05464v1
• Date: Tue, 11 Apr 2023 19:27:18 GMT
• Title: UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Series
• Authors: Patrick Ebel, Vivien Sainte Fare Garnot, Michael Schmitt, Jan Dirk Wegner, Xiao Xiang Zhu
• Abstract summary: We introduce UnCRtainTS, a method for multi-temporal cloud removal combining a novel attention-based architecture. We show how the well-calibrated predicted uncertainties enable a precise control of the reconstruction quality.
• Score: 19.32220113046804
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clouds and haze often occlude optical satellite images, hindering continuous, dense monitoring of the Earth's surface. Although modern deep learning methods can implicitly learn to ignore such occlusions, explicit cloud removal as pre-processing enables manual interpretation and allows training models when only few annotations are available. Cloud removal is challenging due to the wide range of occlusion scenarios -- from scenes partially visible through haze, to completely opaque cloud coverage. Furthermore, integrating reconstructed images in downstream applications would greatly benefit from trustworthy quality assessment. In this paper, we introduce UnCRtainTS, a method for multi-temporal cloud removal combining a novel attention-based architecture, and a formulation for multivariate uncertainty prediction. These two components combined set a new state-of-the-art performance in terms of image reconstruction on two public cloud removal datasets. Additionally, we show how the well-calibrated predicted uncertainties enable a precise control of the reconstruction quality.

Related papers

• Removing cloud shadows from ground-based solar imagery [0.33748750222488655]
  We propose a new method to remove cloud shadows, based on a U-Net architecture, and compare classical supervision with conditional GAN. We evaluate our method on two different imaging modalities, using both real images and a new dataset of synthetic clouds.
  arXiv  Detail & Related papers  (2024-07-18T10:38:24Z)
• Few-shot point cloud reconstruction and denoising via learned Guassian splats renderings and fine-tuned diffusion features [52.62053703535824]
  We propose a method to reconstruct point clouds from few images and to denoise point clouds from their rendering. To improve reconstruction in constraint settings, we regularize the training of a differentiable with hybrid surface and appearance. We demonstrate how these learned filters can be used to remove point cloud noise coming without 3D supervision.
  arXiv  Detail & Related papers  (2024-04-01T13:38:16Z)
• IDF-CR: Iterative Diffusion Process for Divide-and-Conquer Cloud Removal in Remote-sensing Images [55.40601468843028]
  We present an iterative diffusion process for cloud removal (IDF-CR) IDF-CR is divided into two-stage models that address pixel space and latent space. In the latent space stage, the diffusion model transforms low-quality cloud removal into high-quality clean output.
  arXiv  Detail & Related papers  (2024-03-18T15:23:48Z)
• CLiSA: A Hierarchical Hybrid Transformer Model using Orthogonal Cross Attention for Satellite Image Cloud Segmentation [5.178465447325005]
  Deep learning algorithms have emerged as promising approach to solve image segmentation problems. In this paper, we introduce a deep-learning model for effective cloud mask generation named CLiSA - Cloud segmentation via Lipschitz Stable Attention network. We demonstrate both qualitative and quantitative outcomes for multiple satellite image datasets including Landsat-8, Sentinel-2, and Cartosat-2s.
  arXiv  Detail & Related papers  (2023-11-29T09:31:31Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• Curricular Contrastive Regularization for Physics-aware Single Image Dehazing [56.392696439577165]
  We propose a novel curricular contrastive regularization targeted at a consensual contrastive space as opposed to a non-consensual one. Our negatives, which provide better lower-bound constraints, can be assembled from 1) the hazy image, and 2) corresponding restorations by other existing methods. With the unit, as well as curricular contrastive regularization, we establish our dehazing network, named C2PNet.
  arXiv  Detail & Related papers  (2023-03-24T18:18:25Z)
• See Blue Sky: Deep Image Dehaze Using Paired and Unpaired Training Images [73.23687409870656]
  We propose a cycle generative adversarial network to construct a novel end-to-end image dehaze model. We adopt outdoor image datasets to train our model, which includes a set of real-world unpaired image dataset and a set of paired image dataset. Based on the cycle structure, our model adds four different kinds of loss function to constrain the effect including adversarial loss, cycle consistency loss, photorealism loss and paired L1 loss.
  arXiv  Detail & Related papers  (2022-10-14T07:45:33Z)
• Cloud removal Using Atmosphere Model [7.259230333873744]
  Cloud removal is an essential task in remote sensing data analysis. We propose to use scattering model for temporal sequence of images of any scene in the framework of low rank and sparse models. We develop a semi-realistic simulation method to produce cloud cover so that various methods can be quantitatively analysed.
  arXiv  Detail & Related papers  (2022-10-05T01:29:19Z)
• Exploring the Potential of SAR Data for Cloud Removal in Optical Satellite Imagery [41.40522618945897]
  We propose a novel global-local fusion based cloud removal (GLF-CR) algorithm to leverage the complementary information embedded in SAR images. The proposed algorithm can yield high quality cloud-free images and performs favorably against state-of-the-art cloud removal algorithms.
  arXiv  Detail & Related papers  (2022-06-06T18:53:19Z)
• Thick Cloud Removal of Remote Sensing Images Using Temporal Smoothness and Sparsity-Regularized Tensor Optimization [3.65794756599491]
  In remote sensing images, the presence of thick cloud accompanying cloud shadow is a high probability event. A novel thick cloud removal method for remote sensing images based on temporal smoothness and sparsity-regularized tensor optimization is proposed.
  arXiv  Detail & Related papers  (2020-08-11T05:59:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.