Self-supervised Neural Networks for Spectral Snapshot Compressive Imaging

• URL: http://arxiv.org/abs/2108.12654v1
• Date: Sat, 28 Aug 2021 14:17:38 GMT
• Title: Self-supervised Neural Networks for Spectral Snapshot Compressive Imaging
• Authors: Ziyi Meng and Zhenming Yu and Kun Xu and Xin Yuan
• Abstract summary: We consider using untrained neural networks to solve the reconstruction problem of snapshot compressive imaging (SCI) In this paper, inspired by the untrained neural networks such as deep image priors (DIP) and deep decoders, we develop a framework by integrating DIP into the plug-and-play regime, leading to a self-supervised network for spectral SCI reconstruction.
• Score: 15.616674529295366
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We consider using {\bf\em untrained neural networks} to solve the reconstruction problem of snapshot compressive imaging (SCI), which uses a two-dimensional (2D) detector to capture a high-dimensional (usually 3D) data-cube in a compressed manner. Various SCI systems have been built in recent years to capture data such as high-speed videos, hyperspectral images, and the state-of-the-art reconstruction is obtained by the deep neural networks. However, most of these networks are trained in an end-to-end manner by a large amount of corpus with sometimes simulated ground truth, measurement pairs. In this paper, inspired by the untrained neural networks such as deep image priors (DIP) and deep decoders, we develop a framework by integrating DIP into the plug-and-play regime, leading to a self-supervised network for spectral SCI reconstruction. Extensive synthetic and real data results show that the proposed algorithm without training is capable of achieving competitive results to the training based networks. Furthermore, by integrating the proposed method with a pre-trained deep denoising prior, we have achieved state-of-the-art results. {Our code is available at \url{https://github.com/mengziyi64/CASSI-Self-Supervised}.}

Related papers

• JSRNN: Joint Sampling and Reconstruction Neural Networks for High Quality Image Compressed Sensing [8.902545322578925]
  Two sub-networks, which are the sampling sub-network and the reconstruction sub-network, are included in the proposed framework. In the reconstruction sub-network, a cascade network combining stacked denoising autoencoder (SDA) and convolutional neural network (CNN) is designed to reconstruct signals. This framework outperforms many other state-of-the-art methods, especially at low sampling rates.
  arXiv  Detail & Related papers  (2022-11-11T02:20:30Z)
• GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs [49.55919802779889]
  We propose a Graph Convolution based Spatial Propagation Network (GraphCSPN) as a general approach for depth completion. In this work, we leverage convolution neural networks as well as graph neural networks in a complementary way for geometric representation learning. Our method achieves the state-of-the-art performance, especially when compared in the case of using only a few propagation steps.
  arXiv  Detail & Related papers  (2022-10-19T17:56:03Z)
• Is Deep Image Prior in Need of a Good Education? [57.3399060347311]
  Deep image prior was introduced as an effective prior for image reconstruction. Despite its impressive reconstructive properties, the approach is slow when compared to learned or traditional reconstruction techniques. We develop a two-stage learning paradigm to address the computational challenge.
  arXiv  Detail & Related papers  (2021-11-23T15:08:26Z)
• Self-Denoising Neural Networks for Few Shot Learning [66.38505903102373]
  We present a new training scheme that adds noise at multiple stages of an existing neural architecture while simultaneously learning to be robust to this added noise. This architecture, which we call a Self-Denoising Neural Network (SDNN), can be applied easily to most modern convolutional neural architectures.
  arXiv  Detail & Related papers  (2021-10-26T03:28:36Z)
• End to end hyperspectral imaging system with coded compression imaging process [13.00211539170695]
  We present a physics-informed self-supervising CNN method based on a coded aperture spectral imaging system. Our method effectively exploits the spatial-spectral relativization from the coded spectral information and forms a self-supervising system based on the camera quantum effect model.
  arXiv  Detail & Related papers  (2021-09-06T13:39:54Z)
• Compressive spectral image classification using 3D coded convolutional neural network [12.67293744927537]
  This paper develops a novel deep learning HIC approach based on measurements of coded-aperture snapshot spectral imagers (CASSI) A new kind of deep learning strategy, namely 3D coded convolutional neural network (3D-CCNN), is proposed to efficiently solve for the classification problem. The accuracy of classification is effectively improved by exploiting the synergy between the deep learning network and coded apertures.
  arXiv  Detail & Related papers  (2020-09-23T15:05:57Z)
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search [65.79109790446257]
  Recent work has shown that the structure of deep convolutional neural networks can be used as a structured image prior. We propose to search for neural architectures that capture stronger image priors. We search for an improved network by leveraging an existing neural architecture search algorithm.
  arXiv  Detail & Related papers  (2020-08-26T17:59:36Z)
• ESPN: Extremely Sparse Pruned Networks [50.436905934791035]
  We show that a simple iterative mask discovery method can achieve state-of-the-art compression of very deep networks. Our algorithm represents a hybrid approach between single shot network pruning methods and Lottery-Ticket type approaches.
  arXiv  Detail & Related papers  (2020-06-28T23:09:27Z)
• Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation [60.80172153614544]
  Un-trained convolutional neural networks have emerged as highly successful tools for image recovery and restoration. We show that an un-trained convolutional neural network can approximately reconstruct signals and images that are sufficiently structured, from a near minimal number of random measurements.
  arXiv  Detail & Related papers  (2020-05-07T15:57:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.