Sea-Land-Cloud Segmentation in Satellite Hyperspectral Imagery by Deep
Learning
- URL: http://arxiv.org/abs/2310.16210v2
- Date: Thu, 28 Dec 2023 19:29:17 GMT
- Title: Sea-Land-Cloud Segmentation in Satellite Hyperspectral Imagery by Deep
Learning
- Authors: Jon Alvarez Justo, Joseph L. Garrett, Mariana-Iuliana Georgescu, Jesus
Gonzalez-Llorente, Radu Tudor Ionescu, Tor Arne Johansen
- Abstract summary: We train 16 different models, whose codes are made available through our study.
We employ the HYPSO-1 mission as an illustrative case for sea-land-cloud segmentation.
Our lightweight DL model, called 1D-Justo-LiuNet, consistently surpasses state-of-the-art models for sea-land-cloud segmentation.
- Score: 23.20416522322131
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Satellites are increasingly adopting on-board AI for enhanced autonomy
through in-orbit inference. In this context, the use of deep learning (DL)
techniques for segmentation in hyperspectral (HS) satellite imagery offers
advantages for remote sensing applications, and therefore, we train 16
different models, whose codes are made available through our study, which we
consider to be relevant for on-board multi-class segmentation of HS imagery,
focusing on classifying oceanic (sea), terrestrial (land), and cloud
formations. We employ the HYPSO-1 mission as an illustrative case for
sea-land-cloud segmentation, and to demonstrate the utility of the segments, we
introduce a novel sea-land-cloud ranking application scenario. We consider how
to prioritize HS image downlink based on sea, land, and cloud coverage levels
from the segmented images. We comparatively evaluate the models for future
in-orbit deployment, considering performance, parameter count, and inference
time. The models include both shallow and deep models, and after we propose
four new DL models, we demonstrate that segmenting single spectral signatures
(1D) outperforms 3D data processing comprising both spectral (1D) and spatial
(2D) contexts. We conclude that our lightweight DL model, called
1D-Justo-LiuNet, consistently surpasses state-of-the-art models for
sea-land-cloud segmentation, such as U-Net and its variations, in terms of
performance (0.93 accuracy) and parameter count (4,563). However, the 1D models
present longer inference time (15s) in the tested processing architecture,
which seems to be a suboptimal architecture for this purpose. Finally, after
demonstrating that in-orbit segmentation should occur post L1b radiance
calibration rather than on raw data, we also show that reducing spectral
channels down to 3 lowers models' parameter counts and inference time, at the
cost of weaker segmentation performance.
Related papers
- SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks.
To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
arXiv Detail & Related papers (2024-03-25T10:30:22Z) - Exploring DINO: Emergent Properties and Limitations for Synthetic
Aperture Radar Imagery [5.057850174013128]
This study delves into the emergent characteristics of the Self-Distillation with No Labels (DINO) algorithm and its application to Synthetic Aperture Radar (SAR) imagery.
We pre-train a vision transformer (ViT)-based DINO model using unlabeled SAR data, and later fine-tune the model to predict high-resolution land cover maps.
We show that ViT attention maps hold great intrinsic value for remote sensing, and could provide useful inputs to other algorithms.
arXiv Detail & Related papers (2023-10-05T12:48:12Z) - Low-Resource White-Box Semantic Segmentation of Supporting Towers on 3D
Point Clouds via Signature Shape Identification [52.77024349608834]
SCENE-Net is a low-resource white-box model for 3D point cloud semantic segmentation.
Our training time on a laptop is 85min, and our inference time is 20ms.
We release a 40000 Km labeled dataset of rural terrain point clouds and our code implementation.
arXiv Detail & Related papers (2023-06-13T14:36:06Z) - SATR: Zero-Shot Semantic Segmentation of 3D Shapes [74.08209893396271]
We explore the task of zero-shot semantic segmentation of 3D shapes by using large-scale off-the-shelf 2D image recognition models.
We develop the Assignment with Topological Reweighting (SATR) algorithm and evaluate it on ShapeNetPart and our proposed FAUST benchmarks.
SATR achieves state-of-the-art performance and outperforms a baseline algorithm by 1.3% and 4% average mIoU.
arXiv Detail & Related papers (2023-04-11T00:43:16Z) - Guided deep learning by subaperture decomposition: ocean patterns from
SAR imagery [36.922471841100176]
Sentinel 1 SAR wave mode vignettes have made possible to capture many important oceanic and atmospheric phenomena since 2014.
In this study, we propose to apply subaperture decomposition as a preprocessing stage for SAR deep learning models.
arXiv Detail & Related papers (2022-04-09T09:49:05Z) - Simple and Effective Synthesis of Indoor 3D Scenes [78.95697556834536]
We study the problem of immersive 3D indoor scenes from one or more images.
Our aim is to generate high-resolution images and videos from novel viewpoints.
We propose an image-to-image GAN that maps directly from reprojections of incomplete point clouds to full high-resolution RGB-D images.
arXiv Detail & Related papers (2022-04-06T17:54:46Z) - Highly Accurate Dichotomous Image Segmentation [139.79513044546]
A new task called dichotomous image segmentation (DIS) aims to segment highly accurate objects from natural images.
We collect the first large-scale dataset, DIS5K, which contains 5,470 high-resolution (e.g., 2K, 4K or larger) images.
We also introduce a simple intermediate supervision baseline (IS-Net) using both feature-level and mask-level guidance for DIS model training.
arXiv Detail & Related papers (2022-03-06T20:09:19Z) - Hyperspectral Classification Based on Lightweight 3-D-CNN With Transfer
Learning [67.40866334083941]
We propose an end-to-end 3-D lightweight convolutional neural network (CNN) for limited samples-based HSI classification.
Compared with conventional 3-D-CNN models, the proposed 3-D-LWNet has a deeper network structure, less parameters, and lower computation cost.
Our model achieves competitive performance for HSI classification compared to several state-of-the-art methods.
arXiv Detail & Related papers (2020-12-07T03:44:35Z) - Deep Learning based Segmentation of Fish in Noisy Forward Looking MBES
Images [1.5469452301122177]
We build on recent advances in Deep Learning (DL) and Convolutional Neural Networks (CNNs) for semantic segmentation.
We demonstrate an end-to-end approach for a fish/non-fish probability prediction for all range-azimuth positions projected by an imaging sonar.
We show that our model proves the desired performance and has learned to harness the importance of semantic context.
arXiv Detail & Related papers (2020-06-16T09:57:38Z) - A Nearest Neighbor Network to Extract Digital Terrain Models from 3D
Point Clouds [1.6249267147413524]
We present an algorithm that operates on 3D-point clouds and estimates the underlying DTM for the scene using an end-to-end approach.
Our model learns neighborhood information and seamlessly integrates this with point-wise and block-wise global features.
arXiv Detail & Related papers (2020-05-21T15:54:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.