Breaking Spatial Boundaries: Spectral-Domain Registration Guided Hyperspectral and Multispectral Blind Fusion

• URL: http://arxiv.org/abs/2506.20293v1
• Date: Wed, 25 Jun 2025 10:00:51 GMT
• Title: Breaking Spatial Boundaries: Spectral-Domain Registration Guided Hyperspectral and Multispectral Blind Fusion
• Authors: Kunjing Yang, Libin Zheng, Minru Bai, Ting Lu, Leyuan Fang,
• Abstract summary: The blind fusion of unregistered hyperspectral images (HSIs) and multispectral images (MSIs) has attracted growing attention recently.<n>To address the registration challenge, most existing methods employ spatial transformations on the HSI to achieve alignment with the MSI.<n>We propose tackling the registration problem from the spectral domain.
• Score: 14.285239151249193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The blind fusion of unregistered hyperspectral images (HSIs) and multispectral images (MSIs) has attracted growing attention recently. To address the registration challenge, most existing methods employ spatial transformations on the HSI to achieve alignment with the MSI. However, due to the substantial differences in spatial resolution of the images, the performance of these methods is often unsatisfactory. Moreover, the registration process tends to be time-consuming when dealing with large-sized images in remote sensing. To address these issues, we propose tackling the registration problem from the spectral domain. Initially, a lightweight Spectral Prior Learning (SPL) network is developed to extract spectral features from the HSI and enhance the spectral resolution of the MSI. Following this, the obtained image undergoes spatial downsampling to produce the registered HSI. In this process, subspace representation and cyclic training strategy are employed to improve spectral accuracy of the registered HSI obtained. Next, we propose a blind sparse fusion (BSF) method, which utilizes group sparsity regularization to equivalently promote the low-rankness of the image. This approach not only circumvents the need for rank estimation, but also reduces computational complexity. Then, we employ the Proximal Alternating Optimization (PAO) algorithm to solve the BSF model, and present its convergence analysis. Finally, extensive numerical experiments on simulated and real datasets are conducted to verify the effectiveness of our method in registration and fusion. We also demonstrate its efficacy in enhancing classification performance.

Related papers

• A Hybrid Registration and Fusion Method for Hyperspectral Super-resolution [0.913509220304172]
  We propose a hybrid registration and fusion model named RAF-NLRGS. The RAF model incorporates a constrained group sparsity to harness valuable information embedded in the residual data group. We also present the framework of Generalized Gauss-Newton (GGN) algorithm and Proximal Alternating Optimization (PAO)
  arXiv  Detail & Related papers  (2024-07-07T06:36:19Z)
• Physics-Inspired Degradation Models for Hyperspectral Image Fusion [61.743696362028246]
  Most fusion methods solely focus on the fusion algorithm itself and overlook the degradation models. We propose physics-inspired degradation models (PIDM) to model the degradation of LR-HSI and HR-MSI. Our proposed PIDM can boost the fusion performance of existing fusion methods in practical scenarios.
  arXiv  Detail & Related papers  (2024-02-04T09:07:28Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Hyperspectral and Multispectral Image Fusion Using the Conditional Denoising Diffusion Probabilistic Model [18.915369996829984]
  We propose a deep fusion method based on the conditional denoising diffusion probabilistic model, called DDPM-Fus. Experiments conducted on one indoor and two remote sensing datasets show the superiority of the proposed model when compared with other advanced deep learningbased fusion methods.
  arXiv  Detail & Related papers  (2023-07-07T07:08:52Z)
• Spectral Enhanced Rectangle Transformer for Hyperspectral Image Denoising [64.11157141177208]
  We propose a spectral enhanced rectangle Transformer to model the spatial and spectral correlation in hyperspectral images. For the former, we exploit the rectangle self-attention horizontally and vertically to capture the non-local similarity in the spatial domain. For the latter, we design a spectral enhancement module that is capable of extracting global underlying low-rank property of spatial-spectral cubes to suppress noise.
  arXiv  Detail & Related papers  (2023-04-03T09:42:13Z)
• Hyperspectral Image Segmentation based on Graph Processing over Multilayer Networks [51.15952040322895]
  One important task of hyperspectral image (HSI) processing is the extraction of spectral-spatial features. We propose several approaches to HSI segmentation based on M-GSP feature extraction. Our experimental results demonstrate the strength of M-GSP in HSI processing and spectral-spatial information extraction.
  arXiv  Detail & Related papers  (2021-11-29T23:28:18Z)
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction [127.20208645280438]
  Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
  arXiv  Detail & Related papers  (2021-11-15T16:59:48Z)
• SSCAN: A Spatial-spectral Cross Attention Network for Hyperspectral Image Denoising [12.873607414761093]
  We propose a novel HSI denoising network, termed SSCAN, that combines group convolutions and attention modules. We propose a spectral-spatial attention block (SSAB) to exploit the spatial and spectral information in hyperspectral images in an effective manner. The experimental results indicate that the proposed SSCAN outperforms several state-of-the-art HSI denoising algorithms.
  arXiv  Detail & Related papers  (2021-05-23T14:36:17Z)
• Non-local Meets Global: An Iterative Paradigm for Hyperspectral Image Restoration [66.68541690283068]
  We propose a unified paradigm combining the spatial and spectral properties for hyperspectral image restoration. The proposed paradigm enjoys performance superiority from the non-local spatial denoising and light computation complexity. Experiments on HSI denoising, compressed reconstruction, and inpainting tasks, with both simulated and real datasets, demonstrate its superiority.
  arXiv  Detail & Related papers  (2020-10-24T15:53:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.