SST-ReversibleNet: Reversible-prior-based Spectral-Spatial Transformer for Efficient Hyperspectral Image Reconstruction

• URL: http://arxiv.org/abs/2305.04054v1
• Date: Sat, 6 May 2023 14:01:02 GMT
• Title: SST-ReversibleNet: Reversible-prior-based Spectral-Spatial Transformer for Efficient Hyperspectral Image Reconstruction
• Authors: Zeyu Cai, Jian Yu, Ziyu Zhang, Chengqian Jin, Feipeng Da
• Abstract summary: A novel framework called the reversible-prior-based method is proposed. ReversibleNet significantly outperforms state-of-the-art methods on simulated and real HSI datasets.
• Score: 15.233185887461826
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spectral image reconstruction is an important task in snapshot compressed imaging. This paper aims to propose a new end-to-end framework with iterative capabilities similar to a deep unfolding network to improve reconstruction accuracy, independent of optimization conditions, and to reduce the number of parameters. A novel framework called the reversible-prior-based method is proposed. Inspired by the reversibility of the optical path, the reversible-prior-based framework projects the reconstructions back into the measurement space, and then the residuals between the projected data and the real measurements are fed into the network for iteration. The reconstruction subnet in the network then learns the mapping of the residuals to the true values to improve reconstruction accuracy. In addition, a novel spectral-spatial transformer is proposed to account for the global correlation of spectral data in both spatial and spectral dimensions while balancing network depth and computational complexity, in response to the shortcomings of existing transformer-based denoising modules that ignore spatial texture features or learn local spatial features at the expense of global spatial features. Extensive experiments show that our SST-ReversibleNet significantly outperforms state-of-the-art methods on simulated and real HSI datasets, while requiring lower computational and storage costs. https://github.com/caizeyu1992/SST

Related papers

• Empowering Snapshot Compressive Imaging: Spatial-Spectral State Space Model with Across-Scanning and Local Enhancement [51.557804095896174]
  We introduce a State Space Model with Across-Scanning and Local Enhancement, named ASLE-SSM, that employs a Spatial-Spectral SSM for global-local balanced context encoding and cross-channel interaction promoting. Experimental results illustrate ASLE-SSM's superiority over existing state-of-the-art methods, with an inference speed 2.4 times faster than Transformer-based MST and saving 0.12 (M) of parameters.
  arXiv  Detail & Related papers  (2024-08-01T15:14:10Z)
• A Lightweight Recurrent Learning Network for Sustainable Compressed Sensing [27.964167481909588]
  We propose a lightweight but effective deep neural network based on recurrent learning to achieve a sustainable CS system. Our proposed model can achieve a better reconstruction quality than existing state-of-the-art CS algorithms.
  arXiv  Detail & Related papers  (2023-04-23T14:54:15Z)
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
  Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
  arXiv  Detail & Related papers  (2023-03-16T21:06:13Z)
• Learning Detail-Structure Alternative Optimization for Blind Super-Resolution [69.11604249813304]
  We propose an effective and kernel-free network, namely DSSR, which enables recurrent detail-structure alternative optimization without blur kernel prior incorporation for blind SR. In our DSSR, a detail-structure modulation module (DSMM) is built to exploit the interaction and collaboration of image details and structures. Our method achieves the state-of-the-art against existing methods.
  arXiv  Detail & Related papers  (2022-12-03T14:44:17Z)
• Spectral Compressive Imaging Reconstruction Using Convolution and Contextual Transformer [6.929652454131988]
  We propose a hybrid network module, namely CCoT (Contextual Transformer) block, which can acquire the inductive bias ability of transformer simultaneously. We integrate the proposed CCoT block into deep unfolding framework based on the generalized alternating projection algorithm, and further propose the GAP-CT network.
  arXiv  Detail & Related papers  (2022-01-15T06:30:03Z)
• Sparse-View CT Reconstruction using Recurrent Stacked Back Projection [3.91278924473622]
  We introduce a direct-reconstruction method called Recurrent Stacked Back Projection (RSBP) RSBP uses sequentially-acquired backprojections of individual views as input to a recurrent convolutional LSTM network. We demonstrate that RSBP outperforms both post-processing of FBP images and basic MBIR, with a lower computational cost than MBIR.
  arXiv  Detail & Related papers  (2021-12-09T15:44:35Z)
• A New Backbone for Hyperspectral Image Reconstruction [90.48427561874402]
  3D hyperspectral image (HSI) reconstruction refers to inverse process of snapshot compressive imaging. Proposal is for a Spatial/Spectral Invariant Residual U-Net, namely SSI-ResU-Net. We show that SSI-ResU-Net achieves competing performance with over 77.3% reduction in terms of floating-point operations.
  arXiv  Detail & Related papers  (2021-08-17T16:20:51Z)
• Deep Amended Gradient Descent for Efficient Spectral Reconstruction from Single RGB Images [42.26124628784883]
  We propose a compact, efficient, and end-to-end learning-based framework, namely AGD-Net. We first formulate the problem explicitly based on the classic gradient descent algorithm. AGD-Net can improve the reconstruction quality by more than 1.0 dB on average.
  arXiv  Detail & Related papers  (2021-08-12T05:54:09Z)
• Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications [78.63280020581662]
  A novel convolutional neural network (CNN)-based framework is developed for light field reconstruction from a sparse set of views. We demonstrate the high performance and robustness of the proposed framework compared with state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-03-24T08:16:32Z)
• Non-local Meets Global: An Iterative Paradigm for Hyperspectral Image Restoration [66.68541690283068]
  We propose a unified paradigm combining the spatial and spectral properties for hyperspectral image restoration. The proposed paradigm enjoys performance superiority from the non-local spatial denoising and light computation complexity. Experiments on HSI denoising, compressed reconstruction, and inpainting tasks, with both simulated and real datasets, demonstrate its superiority.
  arXiv  Detail & Related papers  (2020-10-24T15:53:56Z)
• High-Order Residual Network for Light Field Super-Resolution [39.93400777363467]
  Plenoptic cameras usually sacrifice the spatial resolution of their SAIss to acquire information from different viewpoints. We propose a novel high-order residual network to learn the geometric features hierarchically from the light field for reconstruction. Our approach enables high-quality reconstruction even in challenging regions and outperforms state-of-the-art single image or LF reconstruction methods with both quantitative measurements and visual evaluation.
  arXiv  Detail & Related papers  (2020-03-29T18:06:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.