Calibrated Hyperspectral Image Reconstruction via Graph-based
Self-Tuning Network
- URL: http://arxiv.org/abs/2112.15362v2
- Date: Tue, 4 Jan 2022 02:46:48 GMT
- Title: Calibrated Hyperspectral Image Reconstruction via Graph-based
Self-Tuning Network
- Authors: Jiamian Wang, Yulun Zhang, Xin Yuan, Ziyi Meng, Zhiqiang Tao
- Abstract summary: Hyperspectral imaging (HSI) has attracted increasing research attention, especially for the ones based on a coded snapshot spectral imaging (CASSI) system.
Existing deep HSI reconstruction models are generally trained on paired data to retrieve original signals upon 2D compressed measurements given by a particular optical hardware mask in CASSI.
This mask-specific training style will lead to a hardware miscalibration issue, which sets up barriers to deploying deep HSI models among different hardware and noisy environments.
We propose a novel Graph-based Self-Tuning ( GST) network to reason uncertainties adapting to varying spatial structures of masks among
- Score: 40.71031760929464
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recently, hyperspectral imaging (HSI) has attracted increasing research
attention, especially for the ones based on a coded aperture snapshot spectral
imaging (CASSI) system. Existing deep HSI reconstruction models are generally
trained on paired data to retrieve original signals upon 2D compressed
measurements given by a particular optical hardware mask in CASSI, during which
the mask largely impacts the reconstruction performance and could work as a
"model hyperparameter" governing on data augmentations. This mask-specific
training style will lead to a hardware miscalibration issue, which sets up
barriers to deploying deep HSI models among different hardware and noisy
environments. To address this challenge, we introduce mask uncertainty for HSI
with a complete variational Bayesian learning treatment and explicitly model it
through a mask decomposition inspired by real hardware. Specifically, we
propose a novel Graph-based Self-Tuning (GST) network to reason uncertainties
adapting to varying spatial structures of masks among different hardware.
Moreover, we develop a bilevel optimization framework to balance HSI
reconstruction and uncertainty estimation, accounting for the hyperparameter
property of masks. Extensive experimental results and model discussions
validate the effectiveness (over 33/30 dB) of the proposed GST method under two
miscalibration scenarios and demonstrate a highly competitive performance
compared with the state-of-the-art well-calibrated methods. Our code and
pre-trained model are available at
https://github.com/Jiamian-Wang/mask_uncertainty_spectral_SCI
Related papers
- Physics-informed DeepCT: Sinogram Wavelet Decomposition Meets Masked Diffusion [9.126628956920904]
Diffusion model shows remarkable potential on sparse-view computed tomography (SVCT) reconstruction.
We propose a Sinogram-based Wavelet random decomposition And Random mask diffusion Model (SWARM) for SVCT reconstruction.
arXiv Detail & Related papers (2025-01-17T03:16:15Z) - SimGen: A Diffusion-Based Framework for Simultaneous Surgical Image and Segmentation Mask Generation [1.9393128408121891]
generative AI models like text-to-image can alleviate data scarcity, incorporating spatial annotations, such as segmentation masks, is crucial for precision-driven surgical applications, simulation, and education.
This study introduces both a novel task and method, SimGen, for Simultaneous Image and Mask Generation.
SimGen is a diffusion model based on the DDPM framework and Residual U-Net, designed to jointly generate high-fidelity surgical images and their corresponding segmentation masks.
arXiv Detail & Related papers (2025-01-15T18:48:38Z) - Mask Factory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentation [70.95380821618711]
Dichotomous Image (DIS) tasks require highly precise annotations.
Current generative models and techniques struggle with the issues of scene deviations, noise-induced errors, and limited training sample variability.
We introduce a novel approach, which provides a scalable solution for generating diverse and precise datasets.
arXiv Detail & Related papers (2024-12-26T06:37:25Z) - Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models.
Recent studies extend the SAM to Few-shot Semantic segmentation (FSS)
We propose a simple yet effective approach based on graph analysis.
arXiv Detail & Related papers (2024-10-09T15:02:28Z) - Cross-Scan Mamba with Masked Training for Robust Spectral Imaging [51.557804095896174]
We propose the Cross-Scanning Mamba, named CS-Mamba, that employs a Spatial-Spectral SSM for global-local balanced context encoding.
Experiment results show that our CS-Mamba achieves state-of-the-art performance and the masked training method can better reconstruct smooth features to improve the visual quality.
arXiv Detail & Related papers (2024-08-01T15:14:10Z) - RARE: Robust Masked Graph Autoencoder [45.485891794905946]
Masked graph autoencoder (MGAE) has emerged as a promising self-supervised graph pre-training (SGP) paradigm.
We propose a novel SGP method termed Robust mAsked gRaph autoEncoder (RARE) to improve the certainty in inferring masked data.
arXiv Detail & Related papers (2023-04-04T03:35:29Z) - Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction [138.04956118993934]
We propose a novel Transformer-based method, coarse-to-fine sparse Transformer (CST)
CST embedding HSI sparsity into deep learning for HSI reconstruction.
In particular, CST uses our proposed spectra-aware screening mechanism (SASM) for coarse patch selecting. Then the selected patches are fed into our customized spectra-aggregation hashing multi-head self-attention (SAH-MSA) for fine pixel clustering and self-similarity capturing.
arXiv Detail & Related papers (2022-03-09T16:17:47Z) - Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image
Reconstruction [127.20208645280438]
Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement.
Modeling the inter-spectra interactions is beneficial for HSI reconstruction.
Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
arXiv Detail & Related papers (2021-11-15T16:59:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.