A New Backbone for Hyperspectral Image Reconstruction
- URL: http://arxiv.org/abs/2108.07739v1
- Date: Tue, 17 Aug 2021 16:20:51 GMT
- Title: A New Backbone for Hyperspectral Image Reconstruction
- Authors: Jiamian Wang, Yulun Zhang, Xin Yuan, Yun Fu, Zhiqiang Tao
- Abstract summary: 3D hyperspectral image (HSI) reconstruction refers to inverse process of snapshot compressive imaging.
Proposal is for a Spatial/Spectral Invariant Residual U-Net, namely SSI-ResU-Net.
We show that SSI-ResU-Net achieves competing performance with over 77.3% reduction in terms of floating-point operations.
- Score: 90.48427561874402
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The study of 3D hyperspectral image (HSI) reconstruction refers to the
inverse process of snapshot compressive imaging, during which the optical
system, e.g., the coded aperture snapshot spectral imaging (CASSI) system,
captures the 3D spatial-spectral signal and encodes it to a 2D measurement.
While numerous sophisticated neural networks have been elaborated for
end-to-end reconstruction, trade-offs still need to be made among performance,
efficiency (training and inference time), and feasibility (the ability of
restoring high resolution HSI on limited GPU memory). This raises a challenge
to design a new baseline to conjointly meet the above requirements. In this
paper, we fill in this blank by proposing a Spatial/Spectral Invariant Residual
U-Net, namely SSI-ResU-Net. It differentiates with U-Net in three folds--1)
scale/spectral-invariant learning, 2) nested residual learning, and 3)
computational efficiency. Benefiting from these three modules, the proposed
SSI-ResU-Net outperforms the current state-of-the-art method TSA-Net by over 3
dB in PSNR and 0.036 in SSIM while only using 2.82% trainable parameters. To
the greatest extent, SSI-ResU-Net achieves competing performance with over
77.3% reduction in terms of floating-point operations (FLOPs), which for the
first time, makes high-resolution HSI reconstruction feasible under practical
application scenarios. Code and pre-trained models are made available at
https://github.com/Jiamian-Wang/HSI_baseline.
Related papers
- Swap-Net: A Memory-Efficient 2.5D Network for Sparse-View 3D Cone Beam CT Reconstruction [13.891441371598546]
Reconstructing 3D cone beam computed tomography (CBCT) images from a limited set of projections is an inverse problem in many imaging applications.
This paper proposes Swap-Net, a memory-efficient 2.5D network for sparse-view 3D CBCT image reconstruction.
arXiv Detail & Related papers (2024-09-29T08:36:34Z) - Coarse-Fine Spectral-Aware Deformable Convolution For Hyperspectral Image Reconstruction [15.537910100051866]
We study the inverse problem of Coded Aperture Snapshot Spectral Imaging (CASSI)
We propose Coarse-Fine Spectral-Aware Deformable Convolution Network (CFSDCN)
Our CFSDCN significantly outperforms previous state-of-the-art (SOTA) methods on both simulated and real HSI datasets.
arXiv Detail & Related papers (2024-06-18T15:15:12Z) - Binarized Spectral Compressive Imaging [59.18636040850608]
Existing deep learning models for hyperspectral image (HSI) reconstruction achieve good performance but require powerful hardwares with enormous memory and computational resources.
We propose a novel method, Binarized Spectral-Redistribution Network (BiSRNet)
BiSRNet is derived by using the proposed techniques to binarize the base model.
arXiv Detail & Related papers (2023-05-17T15:36:08Z) - GLEAM: Greedy Learning for Large-Scale Accelerated MRI Reconstruction [50.248694764703714]
Unrolled neural networks have recently achieved state-of-the-art accelerated MRI reconstruction.
These networks unroll iterative optimization algorithms by alternating between physics-based consistency and neural-network based regularization.
We propose Greedy LEarning for Accelerated MRI reconstruction, an efficient training strategy for high-dimensional imaging settings.
arXiv Detail & Related papers (2022-07-18T06:01:29Z) - HDNet: High-resolution Dual-domain Learning for Spectral Compressive
Imaging [138.04956118993934]
We propose a high-resolution dual-domain learning network (HDNet) for HSI reconstruction.
On the one hand, the proposed HR spatial-spectral attention module with its efficient feature fusion provides continuous and fine pixel-level features.
On the other hand, frequency domain learning (FDL) is introduced for HSI reconstruction to narrow the frequency domain discrepancy.
arXiv Detail & Related papers (2022-03-04T06:37:45Z) - Deep Amended Gradient Descent for Efficient Spectral Reconstruction from
Single RGB Images [42.26124628784883]
We propose a compact, efficient, and end-to-end learning-based framework, namely AGD-Net.
We first formulate the problem explicitly based on the classic gradient descent algorithm.
AGD-Net can improve the reconstruction quality by more than 1.0 dB on average.
arXiv Detail & Related papers (2021-08-12T05:54:09Z) - Fast Hyperspectral Image Recovery via Non-iterative Fusion of
Dual-Camera Compressive Hyperspectral Imaging [22.683482662362337]
Coded aperture snapshot spectral imaging (CASSI) is a promising technique to capture the three-dimensional hyperspectral image (HSI)
Various regularizers have been exploited to reconstruct the 3D data from the 2D measurement.
One feasible solution is to utilize additional information such as the RGB measurement in CASSI.
arXiv Detail & Related papers (2020-12-30T10:29:32Z) - Accurate and Lightweight Image Super-Resolution with Model-Guided Deep
Unfolding Network [63.69237156340457]
We present and advocate an explainable approach toward SISR named model-guided deep unfolding network (MoG-DUN)
MoG-DUN is accurate (producing fewer aliasing artifacts), computationally efficient (with reduced model parameters), and versatile (capable of handling multiple degradations)
The superiority of the proposed MoG-DUN method to existing state-of-theart image methods including RCAN, SRDNF, and SRFBN is substantiated by extensive experiments on several popular datasets and various degradation scenarios.
arXiv Detail & Related papers (2020-09-14T08:23:37Z) - Spatial-Spectral Residual Network for Hyperspectral Image
Super-Resolution [82.1739023587565]
We propose a novel spectral-spatial residual network for hyperspectral image super-resolution (SSRNet)
Our method can effectively explore spatial-spectral information by using 3D convolution instead of 2D convolution, which enables the network to better extract potential information.
In each unit, we employ spatial and temporal separable 3D convolution to extract spatial and spectral information, which not only reduces unaffordable memory usage and high computational cost, but also makes the network easier to train.
arXiv Detail & Related papers (2020-01-14T03:34:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.