SAR-to-RGB Translation with Latent Diffusion for Earth Observation

• URL: http://arxiv.org/abs/2504.11154v1
• Date: Tue, 15 Apr 2025 12:58:30 GMT
• Title: SAR-to-RGB Translation with Latent Diffusion for Earth Observation
• Authors: Kaan Aydin, Joelle Hanna, Damian Borth,
• Abstract summary: We propose a diffusion model (DM)-based approach for SAR-to-RGB translation, generating synthetic optical images from SAR inputs.<n>We evaluate the generated images in downstream tasks, including land cover classification and cloud removal.<n>Our findings highlight the potential of DMs for SAR-to-RGB translation in RS applications where RGB images are missing.
• Score: 5.182014186927255
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Earth observation satellites like Sentinel-1 (S1) and Sentinel-2 (S2) provide complementary remote sensing (RS) data, but S2 images are often unavailable due to cloud cover or data gaps. To address this, we propose a diffusion model (DM)-based approach for SAR-to-RGB translation, generating synthetic optical images from SAR inputs. We explore three different setups: two using Standard Diffusion, which reconstruct S2 images by adding and removing noise (one without and one with class conditioning), and one using Cold Diffusion, which blends S2 with S1 before removing the SAR signal. We evaluate the generated images in downstream tasks, including land cover classification and cloud removal. While generated images may not perfectly replicate real S2 data, they still provide valuable information. Our results show that class conditioning improves classification accuracy, while cloud removal performance remains competitive despite our approach not being optimized for it. Interestingly, despite exhibiting lower perceptual quality, the Cold Diffusion setup performs well in land cover classification, suggesting that traditional quantitative evaluation metrics may not fully reflect the practical utility of generated images. Our findings highlight the potential of DMs for SAR-to-RGB translation in RS applications where RGB images are missing.

Related papers

• Conditional Brownian Bridge Diffusion Model for VHR SAR to Optical Image Translation [5.578820789388206]
  This letter introduces a conditional image-to-image translation approach based on Brownian Bridge Diffusion Model (BBDM) We conducted comprehensive experiments on the MSAW dataset, a paired SAR and optical images collection of 0.5m Very-High-Resolution (VHR)
  arXiv  Detail & Related papers  (2024-08-15T05:43:46Z)
• Joint Image De-noising and Enhancement for Satellite-Based SAR [0.0]
  The reconstructed images from the Synthetic Aperture Radar (SAR) data suffer from multiplicative noise as well as low contrast level.<n>We propose a technique to handle these shortcomings simultaneously.<n>In fact, we combine the de-noising and contrast enhancement processes into a unified algorithm.
  arXiv  Detail & Related papers  (2024-08-06T18:44:16Z)
• Facilitating Advanced Sentinel-2 Analysis Through a Simplified Computation of Nadir BRDF Adjusted Reflectance [0.8971132850029493]
  sen2nbar is a Python package crafted to convert Sentinel-2 (S2) surface reflectance (SR) data to Nadir BRDF Adjusted Reflectance (NBAR) Sen2nbar is developed as a flexible tool that can handle diverse data format requirements.
  arXiv  Detail & Related papers  (2024-04-24T11:26:47Z)
• Non-Visible Light Data Synthesis and Application: A Case Study for Synthetic Aperture Radar Imagery [30.590315753622132]
  We explore the "hidden" ability of large-scale pre-trained image generation models, such as Stable Diffusion and Imagen, in non-visible light domains. We propose a 2-stage low-rank adaptation method, and we call it 2LoRA. In the first stage, the model is adapted using aerial-view regular image data (whose structure matches SAR), followed by the second stage where the base model from the first stage is further adapted using SAR modality data.
  arXiv  Detail & Related papers  (2023-11-29T09:48:01Z)
• Semantic Scene Completion with Cleaner Self [93.99441599791275]
  Semantic Scene Completion (SSC) transforms an image of single-view depth and/or RGB 2D pixels into 3D voxels, each of whose semantic labels are predicted. SSC is a well-known ill-posed problem as the prediction model has to "imagine" what is behind the visible surface, which is usually represented by Truncated Signed Distance Function (TSDF) We use the ground-truth 3D voxels to generate a perfect visible surface, called TSDF-CAD, and then train a "cleaner" SSC model. As the model is noise-free, it is expected to
  arXiv  Detail & Related papers  (2023-03-17T13:50:18Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)
• SAR Despeckling using a Denoising Diffusion Probabilistic Model [52.25981472415249]
  The presence of speckle degrades the image quality and adversely affects the performance of SAR image understanding applications. We introduce SAR-DDPM, a denoising diffusion probabilistic model for SAR despeckling. The proposed method achieves significant improvements in both quantitative and qualitative results over the state-of-the-art despeckling methods.
  arXiv  Detail & Related papers  (2022-06-09T14:00:26Z)
• Residual Contrastive Learning for Joint Demosaicking and Denoising [49.81596361351967]
  We present a novel contrastive learning approach on RAW images, residual contrastive learning (RCL) Our work is built on the assumption that noise contained in each RAW image is signal-dependent. We set a new benchmark for unsupervised JDD tasks with unknown (random) noise variance.
  arXiv  Detail & Related papers  (2021-06-18T11:37:05Z)
• Cloud removal in remote sensing images using generative adversarial networks and SAR-to-optical image translation [0.618778092044887]
  Cloud removal has received much attention due to the wide range of satellite image applications. In this study, we attempt to solve the problem using two generative adversarial networks (GANs) The first translates SAR images into optical images, and the second removes clouds using the translated images of prior GAN.
  arXiv  Detail & Related papers  (2020-12-22T17:19:14Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)
• Self-Supervised Ranking for Representation Learning [108.38993212650577]
  We present a new framework for self-supervised representation learning by formulating it as a ranking problem in an image retrieval context. We train a representation encoder by maximizing average precision (AP) for ranking, where random views of an image are considered positively related. In principle, by using a ranking criterion, we eliminate reliance on object-centric curated datasets.
  arXiv  Detail & Related papers  (2020-10-14T17:24:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.