Pixel-to-Abundance Translation: Conditional Generative Adversarial Networks Based on Patch Transformer for Hyperspectral Unmixing

• URL: http://arxiv.org/abs/2312.13127v1
• Date: Wed, 20 Dec 2023 15:47:21 GMT
• Title: Pixel-to-Abundance Translation: Conditional Generative Adversarial Networks Based on Patch Transformer for Hyperspectral Unmixing
• Authors: Li Wang, Xiaohua Zhang, Longfei Li, Hongyun Meng and Xianghai Cao
• Abstract summary: Spectral unmixing is a significant challenge in hyperspectral image processing. We propose a hyperspectral conditional generative adversarial network (HyperGAN) method as a generic unmixing framework. Experiments on synthetic data and real hyperspectral data achieve impressive results compared to state-of-the-art competitors.
• Score: 12.976092623812757
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spectral unmixing is a significant challenge in hyperspectral image processing. Existing unmixing methods utilize prior knowledge about the abundance distribution to solve the regularization optimization problem, where the difficulty lies in choosing appropriate prior knowledge and solving the complex regularization optimization problem. To solve these problems, we propose a hyperspectral conditional generative adversarial network (HyperGAN) method as a generic unmixing framework, based on the following assumption: the unmixing process from pixel to abundance can be regarded as a transformation of two modalities with an internal specific relationship. The proposed HyperGAN is composed of a generator and discriminator, the former completes the modal conversion from mixed hyperspectral pixel patch to the abundance of corresponding endmember of the central pixel and the latter is used to distinguish whether the distribution and structure of generated abundance are the same as the true ones. We propose hyperspectral image (HSI) Patch Transformer as the main component of the generator, which utilize adaptive attention score to capture the internal pixels correlation of the HSI patch and leverage the spatial-spectral information in a fine-grained way to achieve optimization of the unmixing process. Experiments on synthetic data and real hyperspectral data achieve impressive results compared to state-of-the-art competitors.

Related papers

• QMambaBSR: Burst Image Super-Resolution with Query State Space Model [55.56075874424194]
  Burst super-resolution aims to reconstruct high-resolution images with higher quality and richer details by fusing the sub-pixel information from multiple burst low-resolution frames. In BusrtSR, the key challenge lies in extracting the base frame's content complementary sub-pixel details while simultaneously suppressing high-frequency noise disturbance. We introduce a novel Query Mamba Burst Super-Resolution (QMambaBSR) network, which incorporates a Query State Space Model (QSSM) and Adaptive Up-sampling module (AdaUp)
  arXiv  Detail & Related papers  (2024-08-16T11:15:29Z)
• Hierarchical Homogeneity-Based Superpixel Segmentation: Application to Hyperspectral Image Analysis [11.612069983959985]
  We propose a multiscale superpixel method that is computationally efficient for processing hyperspectral data. The proposed hierarchical approach leads to superpixels of variable sizes but with higher spectral homogeneity. For validation, the proposed homogeneity-based hierarchical method was applied as a preprocessing step in the spectral unmixing and classification tasks.
  arXiv  Detail & Related papers  (2024-07-22T01:20:32Z)
• A Generalized Multiscale Bundle-Based Hyperspectral Sparse Unmixing Algorithm [8.616208042031877]
  In hyperspectral sparse unmixing, a successful approach employs spectral bundles to address the variability of the endmembers in the spatial domain. We generalize a multiscale spatial regularization approach to solve the unmixing problem by incorporating group sparsity-inducing mixed norms.
  arXiv  Detail & Related papers  (2024-01-24T00:37:14Z)
• A Spectral Diffusion Prior for Hyperspectral Image Super-Resolution [14.405562058304074]
  Fusion-based hyperspectral image (HSI) super-resolution aims to produce a high-spatial-resolution HSI by fusing a low-spatial-resolution HSI and a high-spatial-resolution multispectral image. Motivated by the success of diffusion models, we propose a novel spectral diffusion prior for fusion-based HSI super-resolution.
  arXiv  Detail & Related papers  (2023-11-15T13:40:58Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion [144.9653045465908]
  We propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM) Our approach yields promising fusion results in infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2023-03-13T04:06:42Z)
• Decoupled-and-Coupled Networks: Self-Supervised Hyperspectral Image Super-Resolution with Subpixel Fusion [67.35540259040806]
  We propose a subpixel-level HS super-resolution framework by devising a novel decoupled-and-coupled network, called DCNet. As the name suggests, DC-Net first decouples the input into common (or cross-sensor) and sensor-specific components. We append a self-supervised learning module behind the CSU net by guaranteeing the material consistency to enhance the detailed appearances of the restored HS product.
  arXiv  Detail & Related papers  (2022-05-07T23:40:36Z)
• Regularization by Denoising Sub-sampled Newton Method for Spectral CT Multi-Material Decomposition [78.37855832568569]
  We propose to solve a model-based maximum-a-posterior problem to reconstruct multi-materials images with application to spectral CT. In particular, we propose to solve a regularized optimization problem based on a plug-in image-denoising function. We show numerical and experimental results for spectral CT materials decomposition.
  arXiv  Detail & Related papers  (2021-03-25T15:20:10Z)
• Subspace-Based Feature Fusion From Hyperspectral And Multispectral Image For Land Cover Classification [17.705966155216945]
  A feature fusion method from hyperspectral (HS) and multispectral (MS) images for pixel-based classification is proposed. The proposed method first extracts spatial features from the MS image using morphological profiles. An algorithm based on combining alternating optimization (AO) and the alternating direction method of multipliers (ADMM) is developed to solve efficiently the feature fusion problem.
  arXiv  Detail & Related papers  (2021-02-22T17:59:18Z)
• A Novel adaptive optimization of Dual-Tree Complex Wavelet Transform for Medical Image Fusion [0.0]
  multimodal image fusion algorithm based on dual-tree complex wavelet transform (DT-CWT) and adaptive particle swarm optimization (APSO) is proposed. Experiment results show that the proposed method is remarkably better than the method based on particle swarm optimization.
  arXiv  Detail & Related papers  (2020-07-22T15:34:01Z)
• Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery [79.69449412334188]
  In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches. We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data. Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
  arXiv  Detail & Related papers  (2020-05-18T14:25:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.