A lightweight deep learning based cloud detection method for Sentinel-2A imagery fusing multi-scale spectral and spatial features

• URL: http://arxiv.org/abs/2105.00967v1
• Date: Thu, 29 Apr 2021 09:36:42 GMT
• Title: A lightweight deep learning based cloud detection method for Sentinel-2A imagery fusing multi-scale spectral and spatial features
• Authors: Jun Li, Zhaocong Wu, Zhongwen Hu, Canliang Jian, Shaojie Luo, Lichao Mou, Xiao Xiang Zhu and Matthieu Molinier
• Abstract summary: We propose a lightweight network for cloud detection, fusing multi-scale spectral and spatial features (CDFM3SF) CDFM3SF outperforms traditional cloud detection methods and state-of-theart deep learning-based methods in both accuracy and speed.
• Score: 17.914305435378783
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clouds are a very important factor in the availability of optical remote sensing images. Recently, deep learning-based cloud detection methods have surpassed classical methods based on rules and physical models of clouds. However, most of these deep models are very large which limits their applicability and explainability, while other models do not make use of the full spectral information in multi-spectral images such as Sentinel-2. In this paper, we propose a lightweight network for cloud detection, fusing multi-scale spectral and spatial features (CDFM3SF) and tailored for processing all spectral bands in Sentinel- 2A images. The proposed method consists of an encoder and a decoder. In the encoder, three input branches are designed to handle spectral bands at their native resolution and extract multiscale spectral features. Three novel components are designed: a mixed depth-wise separable convolution (MDSC) and a shared and dilated residual block (SDRB) to extract multi-scale spatial features, and a concatenation and sum (CS) operation to fuse multi-scale spectral and spatial features with little calculation and no additional parameters. The decoder of CD-FM3SF outputs three cloud masks at the same resolution as input bands to enhance the supervision information of small, middle and large clouds. To validate the performance of the proposed method, we manually labeled 36 Sentinel-2A scenes evenly distributed over mainland China. The experiment results demonstrate that CD-FM3SF outperforms traditional cloud detection methods and state-of-theart deep learning-based methods in both accuracy and speed.

Related papers

• SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening [14.293042131263924]
  We introduce a spatial-spectral integrated diffusion model for the remote sensing pansharpening task, called SSDiff. SSDiff considers the pansharpening process as the fusion process of spatial and spectral components from the perspective of subspace decomposition.
  arXiv  Detail & Related papers  (2024-04-17T16:30:56Z)
• SS-MAE: Spatial-Spectral Masked Auto-Encoder for Multi-Source Remote Sensing Image Classification [35.52272615695294]
  We propose a spatial-spectral masked auto-encoder (SS-MAE) for HSI and LiDAR/SAR data joint classification. Our SS-MAE fully exploits the spatial and spectral representations of the input data. To complement local features in the training stage, we add two lightweight CNNs for feature extraction.
  arXiv  Detail & Related papers  (2023-11-08T03:54:44Z)
• CD-CTFM: A Lightweight CNN-Transformer Network for Remote Sensing Cloud Detection Fusing Multiscale Features [5.600932842087808]
  A lightweight CNN-Transformer network, CD-CTFM, is proposed to solve the problem. CD-CTFM is based on encoder-decoder architecture and incorporates the attention mechanism. The proposed model is evaluated on two cloud datasets, 38-Cloud and MODIS.
  arXiv  Detail & Related papers  (2023-06-12T15:37:18Z)
• U-TILISE: A Sequence-to-sequence Model for Cloud Removal in Optical Satellite Time Series [22.39321609253005]
  We develop a neural model that can map a cloud-masked input sequence to a cloud-free output sequence. We experimentally evaluate the proposed model on a dataset of satellite Sentinel-2 time series acquired all over Europe. Compared to a standard baseline, it increases the PSNR by 1.8 dB at previously seen locations and by 1.3 dB at unseen locations.
  arXiv  Detail & Related papers  (2023-05-22T17:37:10Z)
• Multimodal Industrial Anomaly Detection via Hybrid Fusion [59.16333340582885]
  We propose a novel multimodal anomaly detection method with hybrid fusion scheme. Our model outperforms the state-of-the-art (SOTA) methods on both detection and segmentation precision on MVTecD-3 AD dataset.
  arXiv  Detail & Related papers  (2023-03-01T15:48:27Z)
• CLONeR: Camera-Lidar Fusion for Occupancy Grid-aided Neural Representations [77.90883737693325]
  This paper proposes CLONeR, which significantly improves upon NeRF by allowing it to model large outdoor driving scenes observed from sparse input sensor views. This is achieved by decoupling occupancy and color learning within the NeRF framework into separate Multi-Layer Perceptrons (MLPs) trained using LiDAR and camera data, respectively. In addition, this paper proposes a novel method to build differentiable 3D Occupancy Grid Maps (OGM) alongside the NeRF model, and leverage this occupancy grid for improved sampling of points along a ray for rendering in metric space.
  arXiv  Detail & Related papers  (2022-09-02T17:44:50Z)
• EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation [62.210091681352914]
  We study multi-sensor fusion for 3D semantic segmentation for many applications, such as autonomous driving and robotics. In this work, we investigate a collaborative fusion scheme called perception-aware multi-sensor fusion (PMF) We propose a two-stream network to extract features from the two modalities separately. The extracted features are fused by effective residual-based fusion modules.
  arXiv  Detail & Related papers  (2021-06-21T10:47:26Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation [30.736361776703568]
  Scene understanding based on LiDAR point cloud is an essential task for autonomous cars to drive safely. Most existing methods simply stack different point attributes/modalities as image channels to increase information capacity. We design FPS-Net, a convolutional fusion network that exploits the uniqueness and discrepancy among the projected image channels for optimal point cloud segmentation.
  arXiv  Detail & Related papers  (2021-03-01T04:08:28Z)
• A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection [89.88222217065858]
  We design a single stream network to use the depth map to guide early fusion and middle fusion between RGB and depth. This model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a $384 times 384$ image.
  arXiv  Detail & Related papers  (2020-07-14T04:40:14Z)
• Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution [82.1739023587565]
  We propose a novel spectral-spatial residual network for hyperspectral image super-resolution (SSRNet) Our method can effectively explore spatial-spectral information by using 3D convolution instead of 2D convolution, which enables the network to better extract potential information. In each unit, we employ spatial and temporal separable 3D convolution to extract spatial and spectral information, which not only reduces unaffordable memory usage and high computational cost, but also makes the network easier to train.
  arXiv  Detail & Related papers  (2020-01-14T03:34:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.