Spectral Compressive Imaging Reconstruction Using Convolution and Contextual Transformer

• URL: http://arxiv.org/abs/2201.05768v4
• Date: Sun, 2 Jul 2023 06:44:05 GMT
• Title: Spectral Compressive Imaging Reconstruction Using Convolution and Contextual Transformer
• Authors: Lishun Wang, Zongliang Wu, Yong Zhong, Xin Yuan
• Abstract summary: We propose a hybrid network module, namely CCoT (Contextual Transformer) block, which can acquire the inductive bias ability of transformer simultaneously. We integrate the proposed CCoT block into deep unfolding framework based on the generalized alternating projection algorithm, and further propose the GAP-CT network.
• Score: 6.929652454131988
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spectral compressive imaging (SCI) is able to encode the high-dimensional hyperspectral image to a 2D measurement, and then uses algorithms to reconstruct the spatio-spectral data-cube. At present, the main bottleneck of SCI is the reconstruction algorithm, and the state-of-the-art (SOTA) reconstruction methods generally face the problem of long reconstruction time and/or poor detail recovery. In this paper, we propose a novel hybrid network module, namely CCoT (Convolution and Contextual Transformer) block, which can acquire the inductive bias ability of convolution and the powerful modeling ability of transformer simultaneously,and is conducive to improving the quality of reconstruction to restore fine details. We integrate the proposed CCoT block into deep unfolding framework based on the generalized alternating projection algorithm, and further propose the GAP-CCoT network. Through the experiments of extensive synthetic and real data, our proposed model achieves higher reconstruction quality ($>$2dB in PSNR on simulated benchmark datasets) and shorter running time than existing SOTA algorithms by a large margin. The code and models are publicly available at https://github.com/ucaswangls/GAP-CCoT.

Related papers

• MsDC-DEQ-Net: Deep Equilibrium Model (DEQ) with Multi-scale Dilated Convolution for Image Compressive Sensing (CS) [0.0]
  Compressive sensing (CS) is a technique that enables the recovery of sparse signals using fewer measurements than traditional sampling methods. We develop an interpretable and concise neural network model for reconstructing natural images using CS. The model, called MsDC-DEQ-Net, exhibits competitive performance compared to state-of-the-art network-based methods.
  arXiv  Detail & Related papers  (2024-01-05T16:25:58Z)
• Latent Diffusion Prior Enhanced Deep Unfolding for Snapshot Spectral Compressive Imaging [17.511583657111792]
  Snapshot spectral imaging reconstruction aims to reconstruct three-dimensional spatial-spectral images from a single-shot two-dimensional compressed measurement. We introduce a generative model, namely the latent diffusion model (LDM), to generate degradation-free prior to deep unfolding method.
  arXiv  Detail & Related papers  (2023-11-24T04:55:20Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Binarized Spectral Compressive Imaging [59.18636040850608]
  Existing deep learning models for hyperspectral image (HSI) reconstruction achieve good performance but require powerful hardwares with enormous memory and computational resources. We propose a novel method, Binarized Spectral-Redistribution Network (BiSRNet) BiSRNet is derived by using the proposed techniques to binarize the base model.
  arXiv  Detail & Related papers  (2023-05-17T15:36:08Z)
• SST-ReversibleNet: Reversible-prior-based Spectral-Spatial Transformer for Efficient Hyperspectral Image Reconstruction [15.233185887461826]
  A novel framework called the reversible-prior-based method is proposed. ReversibleNet significantly outperforms state-of-the-art methods on simulated and real HSI datasets.
  arXiv  Detail & Related papers  (2023-05-06T14:01:02Z)
• Learning Detail-Structure Alternative Optimization for Blind Super-Resolution [69.11604249813304]
  We propose an effective and kernel-free network, namely DSSR, which enables recurrent detail-structure alternative optimization without blur kernel prior incorporation for blind SR. In our DSSR, a detail-structure modulation module (DSMM) is built to exploit the interaction and collaboration of image details and structures. Our method achieves the state-of-the-art against existing methods.
  arXiv  Detail & Related papers  (2022-12-03T14:44:17Z)
• Deep Unfolding of the DBFB Algorithm with Application to ROI CT Imaging with Limited Angular Density [15.143939192429018]
  This paper presents a new method for reconstructing regions of interest (ROI) from a limited number of computed (CT) measurements. Deep methods are fast, and they can reach high reconstruction quality by leveraging information from datasets. We introduce an unfolding neural network called UDBFB designed for ROI reconstruction from limited data.
  arXiv  Detail & Related papers  (2022-09-27T09:10:57Z)
• Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging [142.11622043078867]
  We propose a principled Degradation-Aware Unfolding Framework (DAUF) that estimates parameters from the compressed image and physical mask, and then uses these parameters to control each iteration. By plugging HST into DAUF, we establish the first Transformer-based deep unfolding method, Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST) for HSI reconstruction.
  arXiv  Detail & Related papers  (2022-05-20T11:37:44Z)
• Sparse-View CT Reconstruction using Recurrent Stacked Back Projection [3.91278924473622]
  We introduce a direct-reconstruction method called Recurrent Stacked Back Projection (RSBP) RSBP uses sequentially-acquired backprojections of individual views as input to a recurrent convolutional LSTM network. We demonstrate that RSBP outperforms both post-processing of FBP images and basic MBIR, with a lower computational cost than MBIR.
  arXiv  Detail & Related papers  (2021-12-09T15:44:35Z)
• Over-and-Under Complete Convolutional RNN for MRI Reconstruction [57.95363471940937]
  Recent deep learning-based methods for MR image reconstruction usually leverage a generic auto-encoder architecture. We propose an Over-and-Under Complete Convolu?tional Recurrent Neural Network (OUCR), which consists of an overcomplete and an undercomplete Convolutional Recurrent Neural Network(CRNN) The proposed method achieves significant improvements over the compressed sensing and popular deep learning-based methods with less number of trainable parameters.
  arXiv  Detail & Related papers  (2021-06-16T15:56:34Z)
• Adaptive Gradient Balancing for UndersampledMRI Reconstruction and Image-to-Image Translation [60.663499381212425]
  We enhance the image quality by using a Wasserstein Generative Adversarial Network combined with a novel Adaptive Gradient Balancing technique. In MRI, our method minimizes artifacts, while maintaining a high-quality reconstruction that produces sharper images than other techniques.
  arXiv  Detail & Related papers  (2021-04-05T13:05:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.