Hyperspectral Image Super-Resolution with Spectral Mixup and
Heterogeneous Datasets
- URL: http://arxiv.org/abs/2101.07589v1
- Date: Tue, 19 Jan 2021 12:19:53 GMT
- Title: Hyperspectral Image Super-Resolution with Spectral Mixup and
Heterogeneous Datasets
- Authors: Ke Li, Dengxin Dai, Ender Konukoglu, Luc Van Gool
- Abstract summary: This work studies Hyperspectral image (HSI) super-resolution (SR)
HSI SR is characterized by high-dimensional data and a limited amount of training examples.
This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples.
- Score: 99.92564298432387
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This work studies Hyperspectral image (HSI) super-resolution (SR). HSI SR is
characterized by high-dimensional data and a limited amount of training
examples. This exacerbates the undesirable behaviors of neural networks such as
memorization and sensitivity to out-of-distribution samples. This work
addresses these issues with three contributions. First, we propose a simple,
yet effective data augmentation routine, termed Spectral Mixup, to construct
effective virtual training samples. Second, we observe that HSI SR and RGB
image SR are correlated and develop a novel multi-tasking network to train them
jointly so that the auxiliary task RGB image SR can provide additional
supervision. Finally, we extend the network to a semi-supervised setting so
that it can learn from datasets containing low-resolution HSIs only. With these
contributions, our method is able to learn from heterogeneous datasets and lift
the requirement for having a large amount of HD HSI training samples. Extensive
experiments on four datasets show that our method outperforms existing methods
significantly and underpin the relevance of our contributions. The code of this
work will be released soon.
Related papers
- UnmixingSR: Material-aware Network with Unsupervised Unmixing as Auxiliary Task for Hyperspectral Image Super-resolution [5.167168688234238]
This paper proposes a component-aware hyperspectral image (HIS) super-resolution network called UnmixingSR.
We use the bond between LR abundances and HR abundances to boost the stability of our method in solving SR problems.
Experimental results show that unmixing process as an auxiliary task incorporated into the SR problem is feasible and rational.
arXiv Detail & Related papers (2024-07-09T03:41:02Z) - CSAKD: Knowledge Distillation with Cross Self-Attention for Hyperspectral and Multispectral Image Fusion [9.3350274016294]
This paper introduces a novel knowledge distillation (KD) framework for HR-MSI/LR-HSI fusion to achieve SR of LR-HSI.
To fully exploit the spatial and spectral feature representations of LR-HSI and HR-MSI, we propose a novel Cross Self-Attention (CSA) fusion module.
Our experimental results demonstrate that the student model achieves comparable or superior LR-HSI SR performance.
arXiv Detail & Related papers (2024-06-28T05:25:57Z) - DMSSN: Distilled Mixed Spectral-Spatial Network for Hyperspectral Salient Object Detection [12.823338405434244]
Hyperspectral salient object detection (HSOD) has exhibited remarkable promise across various applications.
Previous methods insufficiently harness the inherent distinctive attributes of hyperspectral images (HSIs) during the feature extraction process.
We propose Distilled Mixed Spectral-Spatial Network (DMSSN), comprising a Distilled Spectral-Spatial Transformer (MSST)
We have created a large-scale HSOD dataset, HSOD-BIT, to tackle the issue of data scarcity in this field.
arXiv Detail & Related papers (2024-03-31T14:04:57Z) - ADASR: An Adversarial Auto-Augmentation Framework for Hyperspectral and
Multispectral Data Fusion [54.668445421149364]
Deep learning-based hyperspectral image (HSI) super-resolution aims to generate high spatial resolution HSI (HR-HSI) by fusing hyperspectral image (HSI) and multispectral image (MSI) with deep neural networks (DNNs)
In this letter, we propose a novel adversarial automatic data augmentation framework ADASR that automatically optimize and augments HSI-MSI sample pairs to enrich data diversity for HSI-MSI fusion.
arXiv Detail & Related papers (2023-10-11T07:30:37Z) - ESSAformer: Efficient Transformer for Hyperspectral Image
Super-resolution [76.7408734079706]
Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation.
We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
arXiv Detail & Related papers (2023-07-26T07:45:14Z) - Spatial-Spectral Feedback Network for Super-Resolution of Hyperspectral
Imagery [11.76638109321532]
High-dimensional and complex spectral patterns in hyperspectral image make it difficult to explore spatial information and spectral information among bands simultaneously.
The number of available hyperspectral training samples is extremely small, which can easily lead to overfitting when training a deep neural network.
We propose a novel Spatial-Spectral Feedback Network (SSFN) to refine low-level representations among local spectral bands with high-level information from global spectral bands.
arXiv Detail & Related papers (2021-03-07T13:28:48Z) - Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral
Super-Resolution [79.97180849505294]
We propose a novel coupled unmixing network with a cross-attention mechanism, CUCaNet, to enhance the spatial resolution of HSI.
Experiments are conducted on three widely-used HS-MS datasets in comparison with state-of-the-art HSI-SR models.
arXiv Detail & Related papers (2020-07-10T08:08:20Z) - Hyperspectral Image Super-resolution via Deep Progressive Zero-centric
Residual Learning [62.52242684874278]
Cross-modality distribution of spatial and spectral information makes the problem challenging.
We propose a novel textitlightweight deep neural network-based framework, namely PZRes-Net.
Our framework learns a high resolution and textitzero-centric residual image, which contains high-frequency spatial details of the scene.
arXiv Detail & Related papers (2020-06-18T06:32:11Z) - Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral
Imagery [79.69449412334188]
In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches.
We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data.
Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
arXiv Detail & Related papers (2020-05-18T14:25:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.