CloudBreaker: Breaking the Cloud Covers of Sentinel-2 Images using Multi-Stage Trained Conditional Flow Matching on Sentinel-1
- URL: http://arxiv.org/abs/2508.03608v1
- Date: Tue, 05 Aug 2025 16:25:18 GMT
- Title: CloudBreaker: Breaking the Cloud Covers of Sentinel-2 Images using Multi-Stage Trained Conditional Flow Matching on Sentinel-1
- Authors: Saleh Sakib Ahmed, Sara Nowreen, M. Sohel Rahman,
- Abstract summary: Cloud cover and nighttime conditions remain significant limitations in satellite-based remote sensing.<n>We propose CloudBreaker, a novel framework that generates high-quality multi-spectral Sentinel-2 signals from Sentinel-1 data.
- Score: 1.0428401220897083
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Cloud cover and nighttime conditions remain significant limitations in satellite-based remote sensing, often restricting the availability and usability of multi-spectral imagery. In contrast, Sentinel-1 radar images are unaffected by cloud cover and can provide consistent data regardless of weather or lighting conditions. To address the challenges of limited satellite imagery, we propose CloudBreaker, a novel framework that generates high-quality multi-spectral Sentinel-2 signals from Sentinel-1 data. This includes the reconstruction of optical (RGB) images as well as critical vegetation and water indices such as NDVI and NDWI.We employed a novel multi-stage training approach based on conditional latent flow matching and, to the best of our knowledge, are the first to integrate cosine scheduling with flow matching. CloudBreaker demonstrates strong performance, achieving a Frechet Inception Distance (FID) score of 0.7432, indicating high fidelity and realism in the generated optical imagery. The model also achieved Structural Similarity Index Measure (SSIM) of 0.6156 for NDWI and 0.6874 for NDVI, indicating a high degree of structural similarity. This establishes CloudBreaker as a promising solution for a wide range of remote sensing applications where multi-spectral data is typically unavailable or unreliable
Related papers
- EC-Diff: Fast and High-Quality Edge-Cloud Collaborative Inference for Diffusion Models [57.059991285047296]
hybrid edge-cloud collaborative framework was recently proposed to realize fast inference and high-quality generation.<n>Excessive cloud denoising prolongs inference time, while insufficient steps cause semantic ambiguity, leading to inconsistency in edge model output.<n>We propose EC-Diff that accelerates cloud inference through gradient-based noise estimation.<n>Our method significantly enhances generation quality compared to edge inference, while achieving up to an average $2times$ speedup in inference compared to cloud inference.
arXiv Detail & Related papers (2025-07-16T07:23:14Z) - FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [63.87313550399871]
Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose Self-supervised Transfer (PST) and FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.
arXiv Detail & Related papers (2025-03-25T15:04:53Z) - A light-weight model to generate NDWI from Sentinel-1 [0.7996307305274611]
We present a deep learning model that can generate Normalized Difference Water Index (NDWI) from Sentinel-1 images.<n>We show the effectiveness of our model, where it demonstrates a high accuracy of 0.9134 and an AUC of 0.8656 to predict the NDWI.
arXiv Detail & Related papers (2025-01-23T03:50:51Z) - DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal from
Optical Satellite Images [27.02507384522271]
This paper presents a novel framework called DiffCR, which leverages conditional guided diffusion with deep convolutional networks for high-performance cloud removal for optical satellite imagery.
We introduce a decoupled encoder for conditional image feature extraction, providing a robust color representation to ensure the close similarity of appearance information between the conditional input and the synthesized output.
arXiv Detail & Related papers (2023-08-08T17:34:28Z) - U-TILISE: A Sequence-to-sequence Model for Cloud Removal in Optical
Satellite Time Series [22.39321609253005]
We develop a neural model that can map a cloud-masked input sequence to a cloud-free output sequence.
We experimentally evaluate the proposed model on a dataset of satellite Sentinel-2 time series acquired all over Europe.
Compared to a standard baseline, it increases the PSNR by 1.8 dB at previously seen locations and by 1.3 dB at unseen locations.
arXiv Detail & Related papers (2023-05-22T17:37:10Z) - Boosting Point Clouds Rendering via Radiance Mapping [49.24193509772339]
We focus on boosting the image quality of point clouds rendering with a compact model design.
We simplify the NeRF representation to a spatial mapping function which only requires single evaluation per pixel.
Our method achieves the state-of-the-art rendering on point clouds, outperforming prior works by notable margins.
arXiv Detail & Related papers (2022-10-27T01:25:57Z) - Self-Configuring nnU-Nets Detect Clouds in Satellite Images [30.46904432868366]
nnU-Nets is a self-reconfigurable framework able to perform meta-learning of a segmentation network over various datasets.
Our experiments, performed over Sentinel-2 and Landsat-8 multispectral images revealed that nnU-Nets deliver state-of-the-art cloud segmentation performance without any manual design.
arXiv Detail & Related papers (2022-10-24T23:39:58Z) - SEN12MS-CR-TS: A Remote Sensing Data Set for Multi-modal Multi-temporal
Cloud Removal [15.459106705735376]
About half of all optical observations collected via spaceborne satellites are affected by haze or clouds.
This work addresses the challenge of optical satellite image reconstruction and cloud removal by proposing SEN12MS-CR-TS.
We propose two models highlighting the benefits and use cases of SEN12MS-CR-TS.
arXiv Detail & Related papers (2022-01-24T11:38:49Z) - Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual
Fly-Throughs [54.41204057689033]
We explore how to leverage neural fields (NeRFs) to build interactive 3D environments from large-scale visual captures spanning buildings or even multiple city blocks collected primarily from drone data.
In contrast to the single object scenes against which NeRFs have been traditionally evaluated, this setting poses multiple challenges.
We introduce a simple clustering algorithm that partitions training images (or rather pixels) into different NeRF submodules that can be trained in parallel.
arXiv Detail & Related papers (2021-12-20T17:40:48Z) - Sentinel-1 and Sentinel-2 Spatio-Temporal Data Fusion for Clouds Removal [51.9654625216266]
A novel method for clouds-corrupted optical image restoration has been presented and developed based on a joint data fusion paradigm.
It is worth highlighting that the Sentinel code and the dataset have been implemented from scratch and made available to interested research for further analysis and investigation.
arXiv Detail & Related papers (2021-06-23T08:15:01Z) - Conditional Variational Image Deraining [158.76814157115223]
Conditional Variational Image Deraining (CVID) network for better deraining performance.
We propose a spatial density estimation (SDE) module to estimate a rain density map for each image.
Experiments on synthesized and real-world datasets show that the proposed CVID network achieves much better performance than previous deterministic methods on image deraining.
arXiv Detail & Related papers (2020-04-23T11:51:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.