C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation

• URL: http://arxiv.org/abs/2411.10788v2
• Date: Sat, 23 Nov 2024 08:25:59 GMT
• Title: C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation
• Authors: Jeonghyeok Do, Jaehyup Lee, Munchurl Kim,
• Abstract summary: C-DiffSET is a framework leveraging pretrained Latent Diffusion Model (LDM) extensively trained on natural images. Remarkably, we find that the pretrained VAE encoder aligns SAR and EO images in the same latent space, even with varying noise levels in SAR inputs.
• Score: 23.63992950769041
• License:
• Abstract: Synthetic Aperture Radar (SAR) imagery provides robust environmental and temporal coverage (e.g., during clouds, seasons, day-night cycles), yet its noise and unique structural patterns pose interpretation challenges, especially for non-experts. SAR-to-EO (Electro-Optical) image translation (SET) has emerged to make SAR images more perceptually interpretable. However, traditional approaches trained from scratch on limited SAR-EO datasets are prone to overfitting. To address these challenges, we introduce Confidence Diffusion for SAR-to-EO Translation, called C-DiffSET, a framework leveraging pretrained Latent Diffusion Model (LDM) extensively trained on natural images, thus enabling effective adaptation to the EO domain. Remarkably, we find that the pretrained VAE encoder aligns SAR and EO images in the same latent space, even with varying noise levels in SAR inputs. To further improve pixel-wise fidelity for SET, we propose a confidence-guided diffusion (C-Diff) loss that mitigates artifacts from temporal discrepancies, such as appearing or disappearing objects, thereby enhancing structural accuracy. C-DiffSET achieves state-of-the-art (SOTA) results on multiple datasets, significantly outperforming the very recent image-to-image translation methods and SET methods with large margins.

Related papers

• Conditional Brownian Bridge Diffusion Model for VHR SAR to Optical Image Translation [5.578820789388206]
  This paper introduces a conditional image-to-image translation approach based on Brownian Bridge Diffusion Model (BBDM) We conducted comprehensive experiments on the MSAW dataset, a paired SAR and optical images collection of 0.5m Very-High-Resolution (VHR)
  arXiv  Detail & Related papers  (2024-08-15T05:43:46Z)
• Accelerating Diffusion for SAR-to-Optical Image Translation via Adversarial Consistency Distillation [5.234109158596138]
  We propose a new training framework for SAR-to-optical image translation. Our method employs consistency distillation to reduce iterative inference steps and integrates adversarial learning to ensure image clarity and minimize color shifts. The results demonstrate that our approach significantly improves inference speed by 131 times while maintaining the visual quality of the generated images.
  arXiv  Detail & Related papers  (2024-07-08T16:36:12Z)
• SAR Despeckling via Regional Denoising Diffusion Probabilistic Model [6.154796320245652]
  Region Denoising Diffusion Probabilistic Model (R-DDPM) based on generative models. This paper introduces a novel despeckling approach termed Region Denoising Diffusion Probabilistic Model (R-DDPM) based on generative models.
  arXiv  Detail & Related papers  (2024-01-06T04:34:46Z)
• DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [55.48770333927732]
  We propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection. It consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion's denoising network, and a feature-space pre-trained feature extractor. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-12-11T18:38:28Z)
• Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction [75.91471250967703]
  We introduce a novel sampling framework called Steerable Conditional Diffusion. This framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. We achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities.
  arXiv  Detail & Related papers  (2023-08-28T08:47:06Z)
• Improved Flood Insights: Diffusion-Based SAR to EO Image Translation [4.994315051443544]
  This paper introduces a novel framework, Diffusion-Based SAR to EO Image Translation (DSE) The DSE framework converts SAR images into EO images, thereby enhancing the interpretability of flood insights for humans. Experimental results on the Sen1Floods11 and SEN12-FLOOD datasets confirm that the DSE framework not only delivers enhanced visual information but also improves performance.
  arXiv  Detail & Related papers  (2023-07-14T02:19:23Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for Hyperspectral Image Restoration [103.79030498369319]
  Self-supervised diffusion model for hyperspectral image restoration is proposed. textttDDS2M enjoys stronger ability to generalization compared to existing diffusion-based methods. Experiments on HSI denoising, noisy HSI completion and super-resolution on a variety of HSIs demonstrate textttDDS2M's superiority over the existing task-specific state-of-the-arts.
  arXiv  Detail & Related papers  (2023-03-12T14:57:04Z)
• SAR Despeckling using a Denoising Diffusion Probabilistic Model [52.25981472415249]
  The presence of speckle degrades the image quality and adversely affects the performance of SAR image understanding applications. We introduce SAR-DDPM, a denoising diffusion probabilistic model for SAR despeckling. The proposed method achieves significant improvements in both quantitative and qualitative results over the state-of-the-art despeckling methods.
  arXiv  Detail & Related papers  (2022-06-09T14:00:26Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)
• Speckle2Void: Deep Self-Supervised SAR Despeckling with Blind-Spot Convolutional Neural Networks [30.410981386006394]
  despeckling is a crucial preliminary step in scene analysis algorithms. Recent success of deep learning envisions a new generation of despeckling techniques. We propose a self-supervised Bayesian despeckling method.
  arXiv  Detail & Related papers  (2020-07-04T11:38:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.