Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks

• URL: http://arxiv.org/abs/2307.01990v1
• Date: Wed, 5 Jul 2023 02:45:44 GMT
• Title: Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks
• Authors: Kai Feng, Yongqiang Zhao, Seong G. Kong, and Haijin Zeng
• Abstract summary: This paper presents a deep learning-based spectral demosaicing technique trained in an unsupervised manner. The proposed method outperforms conventional unsupervised methods in terms of spatial distortion suppression, spectral fidelity, robustness, and computational cost.
• Score: 6.7433262627741914
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a deep learning-based spectral demosaicing technique trained in an unsupervised manner. Many existing deep learning-based techniques relying on supervised learning with synthetic images, often underperform on real-world images especially when the number of spectral bands increases. According to the characteristics of the spectral mosaic image, this paper proposes a mosaic loss function, the corresponding model structure, a transformation strategy, and an early stopping strategy, which form a complete unsupervised spectral demosaicing framework. A challenge in real-world spectral demosaicing is inconsistency between the model parameters and the computational resources of the imager. We reduce the complexity and parameters of the spectral attention module by dividing the spectral attention tensor into spectral attention matrices in the spatial dimension and spectral attention vector in the channel dimension, which is more suitable for unsupervised framework. This paper also presents Mosaic25, a real 25-band hyperspectral mosaic image dataset of various objects, illuminations, and materials for benchmarking. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method outperforms conventional unsupervised methods in terms of spatial distortion suppression, spectral fidelity, robustness, and computational cost.

Related papers

• Spectral Graph Reasoning Network for Hyperspectral Image Classification [0.43512163406551996]
  Convolutional neural networks (CNNs) have achieved remarkable performance in hyperspectral image (HSI) classification. We propose a spectral graph reasoning network (SGR) learning framework comprising two crucial modules. Experiments on two HSI datasets demonstrate that the proposed architecture can significantly improve the classification accuracy.
  arXiv  Detail & Related papers  (2024-07-02T20:29:23Z)
• Physics-Inspired Degradation Models for Hyperspectral Image Fusion [61.743696362028246]
  Most fusion methods solely focus on the fusion algorithm itself and overlook the degradation models. We propose physics-inspired degradation models (PIDM) to model the degradation of LR-HSI and HR-MSI. Our proposed PIDM can boost the fusion performance of existing fusion methods in practical scenarios.
  arXiv  Detail & Related papers  (2024-02-04T09:07:28Z)
• Datacube segmentation via Deep Spectral Clustering [76.48544221010424]
  Extended Vision techniques often pose a challenge in their interpretation. The huge dimensionality of data cube spectra poses a complex task in its statistical interpretation. In this paper, we explore the possibility of applying unsupervised clustering methods in encoded space. A statistical dimensional reduction is performed by an ad hoc trained (Variational) AutoEncoder, while the clustering process is performed by a (learnable) iterative K-Means clustering algorithm.
  arXiv  Detail & Related papers  (2024-01-31T09:31:28Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction [127.20208645280438]
  Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
  arXiv  Detail & Related papers  (2021-11-15T16:59:48Z)
• Spectral Splitting and Aggregation Network for Hyperspectral Face Super-Resolution [82.59267937569213]
  High-resolution (HR) hyperspectral face image plays an important role in face related computer vision tasks under uncontrolled conditions. In this paper, we investigate how to adapt the deep learning techniques to hyperspectral face image super-resolution. We present a spectral splitting and aggregation network (SSANet) for HFSR with limited training samples.
  arXiv  Detail & Related papers  (2021-08-31T02:13:00Z)
• LADMM-Net: An Unrolled Deep Network For Spectral Image Fusion From Compressive Data [6.230751621285322]
  Hyperspectral (HS) and multispectral (MS) image fusion aims at estimating a high-resolution spectral image from a low-spatial-resolution HS image and a low-spectral-resolution MS image. In this work, a deep learning architecture under the algorithm unrolling approach is proposed for solving the fusion problem from HS and MS compressive measurements.
  arXiv  Detail & Related papers  (2021-03-01T12:04:42Z)
• Spectral Response Function Guided Deep Optimization-driven Network for Spectral Super-resolution [20.014293172511074]
  This paper proposes an optimization-driven convolutional neural network (CNN) with a deep spatial-spectral prior. Experiments on two types of datasets, including natural and remote sensing images, demonstrate the spectral enhancement effect of the proposed method.
  arXiv  Detail & Related papers  (2020-11-19T07:52:45Z)
• Spatial-Spectral Manifold Embedding of Hyperspectral Data [43.479889860715275]
  We propose a novel hyperspectral embedding approach by simultaneously considering spatial and spectral information. spatial-spectral manifold embedding (SSME) models the spatial and spectral information jointly in a patch-based fashion. SSME not only learns the spectral embedding by using the adjacency matrix obtained by similarity measurement between spectral signatures, but also models the spatial neighbours of a target pixel in hyperspectral scene.
  arXiv  Detail & Related papers  (2020-07-17T05:40:27Z)
• Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery [79.69449412334188]
  In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches. We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data. Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
  arXiv  Detail & Related papers  (2020-05-18T14:25:50Z)
• Hyperspectral Image Classification Based on Sparse Modeling of Spectral Blocks [6.99674326582747]
  A sparse modeling framework is proposed for hyperspectral image classification. The proposed method leads to a robust sparse modeling of hyperspectral images and improves the classification accuracy.
  arXiv  Detail & Related papers  (2020-05-17T08:18:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.