CBEN -- A Multimodal Machine Learning Dataset for Cloud Robust Remote Sensing Image Understanding

• URL: http://arxiv.org/abs/2602.12652v1
• Date: Fri, 13 Feb 2026 06:24:55 GMT
• Title: CBEN -- A Multimodal Machine Learning Dataset for Cloud Robust Remote Sensing Image Understanding
• Authors: Marco Stricker, Masakazu Iwamura, Koichi Kise,
• Abstract summary: Cloudless analysis is often performed where cloudy images are excluded from machine learning datasets and methods.<n>Cloud robust methods can be achieved by combining optical data with radar, a modality unaffected by clouds.<n>We show that state-of-the-art methods trained on combined clear-sky optical and radar imagery suffer performance drops of 23-33 percentage points when evaluated on cloudy images.
• Score: 4.405830705915443
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clouds are a common phenomenon that distorts optical satellite imagery, which poses a challenge for remote sensing. However, in the literature cloudless analysis is often performed where cloudy images are excluded from machine learning datasets and methods. Such an approach cannot be applied to time sensitive applications, e.g., during natural disasters. A possible solution is to apply cloud removal as a preprocessing step to ensure that cloudfree solutions are not failing under such conditions. But cloud removal methods are still actively researched and suffer from drawbacks, such as generated visual artifacts. Therefore, it is desirable to develop cloud robust methods that are less affected by cloudy weather. Cloud robust methods can be achieved by combining optical data with radar, a modality unaffected by clouds. While many datasets for machine learning combine optical and radar data, most researchers exclude cloudy images. We identify this exclusion from machine learning training and evaluation as a limitation that reduces applicability to cloudy scenarios. To investigate this, we assembled a dataset, named CloudyBigEarthNet (CBEN), of paired optical and radar images with cloud occlusion for training and evaluation. Using average precision (AP) as the evaluation metric, we show that state-of-the-art methods trained on combined clear-sky optical and radar imagery suffer performance drops of 23-33 percentage points when evaluated on cloudy images. We then adapt these methods to cloudy optical data during training, achieving relative improvement of 17.2-28.7 percentage points on cloudy test cases compared with the original approaches. Code and dataset are publicly available at: https://github.com/mstricker13/CBEN

Related papers

• Not every day is a sunny day: Synthetic cloud injection for deep land cover segmentation robustness evaluation across data sources [1.5052861873036498]
  Supervised deep learning for land cover semantic segmentation (LCS) relies on labeled satellite data.<n>Most existing Sentinel-2 datasets are cloud-free, which limits their usefulness in tropical regions where clouds are common.<n>We develop a cloud injection algorithm that simulates realistic cloud cover, allowing us to test how Sentinel-1 radar data can fill in the gaps caused by cloud-obstructed optical imagery.
  arXiv  Detail & Related papers  (2025-10-03T13:51:35Z)
• Weakly Supervised Cloud Detection Combining Spectral Features and Multi-Scale Deep Network [12.520904004953344]
  We propose a weakly supervised cloud detection method that combines spectral features and multi-scale scene-level deep network (SpecMCD) to obtain highly accurate pixel-level cloud masks.<n>The F1-score of the proposed SpecMCD method shows an improvement of over 7.82%, highlighting the superiority and potential of the SpecMCD method for cloud detection.
  arXiv  Detail & Related papers  (2025-10-01T08:32:49Z)
• Few-shot point cloud reconstruction and denoising via learned Guassian splats renderings and fine-tuned diffusion features [52.62053703535824]
  We propose a method to reconstruct point clouds from few images and to denoise point clouds from their rendering. To improve reconstruction in constraint settings, we regularize the training of a differentiable with hybrid surface and appearance. We demonstrate how these learned filters can be used to remove point cloud noise coming without 3D supervision.
  arXiv  Detail & Related papers  (2024-04-01T13:38:16Z)
• IDF-CR: Iterative Diffusion Process for Divide-and-Conquer Cloud Removal in Remote-sensing Images [55.40601468843028]
  We present an iterative diffusion process for cloud removal (IDF-CR) IDF-CR is divided into two-stage models that address pixel space and latent space. In the latent space stage, the diffusion model transforms low-quality cloud removal into high-quality clean output.
  arXiv  Detail & Related papers  (2024-03-18T15:23:48Z)
• Learning Real-World Image De-Weathering with Imperfect Supervision [57.748585821252824]
  Existing real-world de-weathering datasets often exhibit inconsistent illumination, position, and textures between the ground-truth images and the input degraded images. We develop a Consistent Label Constructor (CLC) to generate a pseudo-label as consistent as possible with the input degraded image. We combine the original imperfect labels and pseudo-labels to jointly supervise the de-weathering model by the proposed Information Allocation Strategy.
  arXiv  Detail & Related papers  (2023-10-23T14:02:57Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Series [19.32220113046804]
  We introduce UnCRtainTS, a method for multi-temporal cloud removal combining a novel attention-based architecture. We show how the well-calibrated predicted uncertainties enable a precise control of the reconstruction quality.
  arXiv  Detail & Related papers  (2023-04-11T19:27:18Z)
• Boosting Point Clouds Rendering via Radiance Mapping [49.24193509772339]
  We focus on boosting the image quality of point clouds rendering with a compact model design. We simplify the NeRF representation to a spatial mapping function which only requires single evaluation per pixel. Our method achieves the state-of-the-art rendering on point clouds, outperforming prior works by notable margins.
  arXiv  Detail & Related papers  (2022-10-27T01:25:57Z)
• Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Survey [104.71816962689296]
  Unsupervised point cloud representation learning has attracted increasing attention due to the constraint in large-scale point cloud labelling. This paper provides a comprehensive review of unsupervised point cloud representation learning using deep neural networks.
  arXiv  Detail & Related papers  (2022-02-28T07:46:05Z)
• Cloud detection machine learning algorithms for PROBA-V [6.950862982117125]
  The objective of the algorithms presented in this paper is to detect clouds accurately providing a cloud flag per pixel. The effectiveness of the proposed method is successfully illustrated using a large number of real Proba-V images.
  arXiv  Detail & Related papers  (2020-12-09T18:23:59Z)
• Thick Cloud Removal of Remote Sensing Images Using Temporal Smoothness and Sparsity-Regularized Tensor Optimization [3.65794756599491]
  In remote sensing images, the presence of thick cloud accompanying cloud shadow is a high probability event. A novel thick cloud removal method for remote sensing images based on temporal smoothness and sparsity-regularized tensor optimization is proposed.
  arXiv  Detail & Related papers  (2020-08-11T05:59:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.