DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal from Optical Satellite Images

• URL: http://arxiv.org/abs/2308.04417v1
• Date: Tue, 8 Aug 2023 17:34:28 GMT
• Title: DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal from Optical Satellite Images
• Authors: Xuechao Zou, Kai Li, Junliang Xing, Yu Zhang, Shiying Wang, Lei Jin, and Pin Tao
• Abstract summary: This paper presents a novel framework called DiffCR, which leverages conditional guided diffusion with deep convolutional networks for high-performance cloud removal for optical satellite imagery. We introduce a decoupled encoder for conditional image feature extraction, providing a robust color representation to ensure the close similarity of appearance information between the conditional input and the synthesized output.
• Score: 27.02507384522271
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical satellite images are a critical data source; however, cloud cover often compromises their quality, hindering image applications and analysis. Consequently, effectively removing clouds from optical satellite images has emerged as a prominent research direction. While recent advancements in cloud removal primarily rely on generative adversarial networks, which may yield suboptimal image quality, diffusion models have demonstrated remarkable success in diverse image-generation tasks, showcasing their potential in addressing this challenge. This paper presents a novel framework called DiffCR, which leverages conditional guided diffusion with deep convolutional networks for high-performance cloud removal for optical satellite imagery. Specifically, we introduce a decoupled encoder for conditional image feature extraction, providing a robust color representation to ensure the close similarity of appearance information between the conditional input and the synthesized output. Moreover, we propose a novel and efficient time and condition fusion block within the cloud removal model to accurately simulate the correspondence between the appearance in the conditional image and the target image at a low computational cost. Extensive experimental evaluations on two commonly used benchmark datasets demonstrate that DiffCR consistently achieves state-of-the-art performance on all metrics, with parameter and computational complexities amounting to only 5.1% and 5.4%, respectively, of those previous best methods. The source code, pre-trained models, and all the experimental results will be publicly available at https://github.com/XavierJiezou/DiffCR upon the paper's acceptance of this work.

Related papers

• Edge-Cloud Collaborative Satellite Image Analysis for Efficient Man-Made Structure Recognition [2.110762118285028]
  The paper presents a new satellite image processing architecture combining edge and cloud computing. By employing lightweight models at the edge, the system initially identifies potential man-made structures from satellite imagery. These identified images are then transmitted to the cloud, where a more complex model refines the classification.
  arXiv  Detail & Related papers  (2024-10-08T03:31:32Z)
• IDF-CR: Iterative Diffusion Process for Divide-and-Conquer Cloud Removal in Remote-sensing Images [55.40601468843028]
  We present an iterative diffusion process for cloud removal (IDF-CR) IDF-CR is divided into two-stage models that address pixel space and latent space. In the latent space stage, the diffusion model transforms low-quality cloud removal into high-quality clean output.
  arXiv  Detail & Related papers  (2024-03-18T15:23:48Z)
• Diffusion Enhancement for Cloud Removal in Ultra-Resolution Remote Sensing Imagery [48.14610248492785]
  Cloud layers severely compromise the quality and effectiveness of optical remote sensing (RS) images. Existing deep-learning (DL)-based Cloud Removal (CR) techniques encounter difficulties in accurately reconstructing the original visual authenticity and detailed semantic content of the images. This work proposes enhancements at the data and methodology fronts to tackle this challenge.
  arXiv  Detail & Related papers  (2024-01-25T13:14:17Z)
• CLiSA: A Hierarchical Hybrid Transformer Model using Orthogonal Cross Attention for Satellite Image Cloud Segmentation [5.178465447325005]
  Deep learning algorithms have emerged as promising approach to solve image segmentation problems. In this paper, we introduce a deep-learning model for effective cloud mask generation named CLiSA - Cloud segmentation via Lipschitz Stable Attention network. We demonstrate both qualitative and quantitative outcomes for multiple satellite image datasets including Landsat-8, Sentinel-2, and Cartosat-2s.
  arXiv  Detail & Related papers  (2023-11-29T09:31:31Z)
• Deep Equilibrium Diffusion Restoration with Parallel Sampling [120.15039525209106]
  Diffusion model-based image restoration (IR) aims to use diffusion models to recover high-quality (HQ) images from degraded images, achieving promising performance. Most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs. In this work, we aim to rethink the diffusion model-based IR models through a different perspective, i.e., a deep equilibrium (DEQ) fixed point system, called DeqIR.
  arXiv  Detail & Related papers  (2023-11-20T08:27:56Z)
• ExposureDiffusion: Learning to Expose for Low-light Image Enhancement [87.08496758469835]
  This work addresses the issue by seamlessly integrating a diffusion model with a physics-based exposure model. Our method obtains significantly improved performance and reduced inference time compared with vanilla diffusion models. The proposed framework can work with both real-paired datasets, SOTA noise models, and different backbone networks.
  arXiv  Detail & Related papers  (2023-07-15T04:48:35Z)
• Denoising Diffusion Models for Plug-and-Play Image Restoration [135.6359475784627]
  This paper proposes DiffPIR, which integrates the traditional plug-and-play method into the diffusion sampling framework. Compared to plug-and-play IR methods that rely on discriminative Gaussian denoisers, DiffPIR is expected to inherit the generative ability of diffusion models.
  arXiv  Detail & Related papers  (2023-05-15T20:24:38Z)
• Dual-Scale Single Image Dehazing Via Neural Augmentation [29.019279446792623]
  A novel single image dehazing algorithm is introduced by combining model-based and data-driven approaches. Results indicate that the proposed algorithm can remove haze well from real-world and synthetic hazy images.
  arXiv  Detail & Related papers  (2022-09-13T11:56:03Z)
• ART-SS: An Adaptive Rejection Technique for Semi-Supervised restoration for adverse weather-affected images [24.03416814412226]
  We study the effect of unlabeled data on the performance of an SSR method. We develop a technique that rejects the unlabeled images that degrade the performance.
  arXiv  Detail & Related papers  (2022-03-17T12:00:31Z)
• Unpaired Quad-Path Cycle Consistent Adversarial Networks for Single Image Defogging [16.59494337699748]
  We develop a novel generative adversarial network, called quad-path cycle consistent adversarial network (QPC-Net) for single image defogging. QPC-Net consists of a Fog2Fogfree block and a Fogfree2Fog block. We show that QPC-Net outperforms state-of-the-art defogging methods in terms of quantitative accuracy and subjective visual quality.
  arXiv  Detail & Related papers  (2022-02-19T09:00:37Z)
• FD-GAN: Generative Adversarial Networks with Fusion-discriminator for Single Image Dehazing [48.65974971543703]
  We propose a fully end-to-end Generative Adversarial Networks with Fusion-discriminator (FD-GAN) for image dehazing. Our model can generator more natural and realistic dehazed images with less color distortion and fewer artifacts. Experiments have shown that our method reaches state-of-the-art performance on both public synthetic datasets and real-world images.
  arXiv  Detail & Related papers  (2020-01-20T04:36:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.