Dense Deep Unfolding Network with 3D-CNN Prior for Snapshot Compressive Imaging

• URL: http://arxiv.org/abs/2109.06548v1
• Date: Tue, 14 Sep 2021 09:42:42 GMT
• Title: Dense Deep Unfolding Network with 3D-CNN Prior for Snapshot Compressive Imaging
• Authors: Zhuoyuan Wu, Jian Zhang, Chong Mou
• Abstract summary: Snapshot imaging (SCI) aims to record three-dimensional signals via a two-dimensional camera. We present a novel dense deep unfolding network (DUN) with 3D-CNN prior for SCI. In order to promote network adaption, we propose a dense feature map compressive (DFMA) module.
• Score: 6.289143409131908
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Snapshot compressive imaging (SCI) aims to record three-dimensional signals via a two-dimensional camera. For the sake of building a fast and accurate SCI recovery algorithm, we incorporate the interpretability of model-based methods and the speed of learning-based ones and present a novel dense deep unfolding network (DUN) with 3D-CNN prior for SCI, where each phase is unrolled from an iteration of Half-Quadratic Splitting (HQS). To better exploit the spatial-temporal correlation among frames and address the problem of information loss between adjacent phases in existing DUNs, we propose to adopt the 3D-CNN prior in our proximal mapping module and develop a novel dense feature map (DFM) strategy, respectively. Besides, in order to promote network robustness, we further propose a dense feature map adaption (DFMA) module to allow inter-phase information to fuse adaptively. All the parameters are learned in an end-to-end fashion. Extensive experiments on simulation data and real data verify the superiority of our method. The source code is available at https://github.com/jianzhangcs/SCI3D.

Related papers

• Efficient Physics-Based Learned Reconstruction Methods for Real-Time 3D Near-Field MIMO Radar Imaging [0.0]
  Near-field multiple-input multiple-output (MIMO) radar imaging systems have recently gained significant attention. In this paper, we develop novel non-iterative deep learning-based reconstruction methods for real-time near-field imaging. The goal is to achieve high image quality with low computational cost at settings.
  arXiv  Detail & Related papers  (2023-12-28T11:05:36Z)
• SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and Quasi-Planar Segmentation [53.83313235792596]
  We present a new methodology for real-time semantic mapping from RGB-D sequences. It combines a 2D neural network and a 3D network based on a SLAM system with 3D occupancy mapping. Our system achieves state-of-the-art semantic mapping quality within 2D-3D networks-based systems.
  arXiv  Detail & Related papers  (2023-06-28T22:36:44Z)
• Unfolding Framework with Prior of Convolution-Transformer Mixture and Uncertainty Estimation for Video Snapshot Compressive Imaging [7.601695814245209]
  We consider the problem of video snapshot compressive imaging (SCI), where sequential high-speed frames are modulated by different masks and captured by a single measurement. By combining optimization algorithms and neural networks, deep unfolding networks (DUNs) score tremendous achievements in solving inverse problems.
  arXiv  Detail & Related papers  (2023-06-20T06:25:48Z)
• GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs [49.55919802779889]
  We propose a Graph Convolution based Spatial Propagation Network (GraphCSPN) as a general approach for depth completion. In this work, we leverage convolution neural networks as well as graph neural networks in a complementary way for geometric representation learning. Our method achieves the state-of-the-art performance, especially when compared in the case of using only a few propagation steps.
  arXiv  Detail & Related papers  (2022-10-19T17:56:03Z)
• Focal Sparse Convolutional Networks for 3D Object Detection [121.45950754511021]
  We introduce two new modules to enhance the capability of Sparse CNNs. They are focal sparse convolution (Focals Conv) and its multi-modal variant of focal sparse convolution with fusion. For the first time, we show that spatially learnable sparsity in sparse convolution is essential for sophisticated 3D object detection.
  arXiv  Detail & Related papers  (2022-04-26T17:34:10Z)
• Similarity-Aware Fusion Network for 3D Semantic Segmentation [87.51314162700315]
  We propose a similarity-aware fusion network (SAFNet) to adaptively fuse 2D images and 3D point clouds for 3D semantic segmentation. We employ a late fusion strategy where we first learn the geometric and contextual similarities between the input and back-projected (from 2D pixels) point clouds. We show that SAFNet significantly outperforms existing state-of-the-art fusion-based approaches across various data integrity.
  arXiv  Detail & Related papers  (2021-07-04T09:28:18Z)
• Improved Brain Age Estimation with Slice-based Set Networks [18.272915375351914]
  We propose a new architecture for BrainAGE prediction. The proposed architecture works by encoding each 2D slice in an MRI with a deep 2D-CNN model. Next, it combines the information from these 2D-slice encodings using set networks or permutation invariant layers. Experiments on the BrainAGE prediction problem, using the UK Biobank dataset, showed that the model with the permutation invariant layers trains faster and provides better predictions compared to other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-02-08T18:54:15Z)
• Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes [77.6741486264257]
  We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs. We show that our representation is 2-3 orders of magnitude more efficient in terms of rendering speed compared to previous works.
  arXiv  Detail & Related papers  (2021-01-26T18:50:22Z)
• Reinforced Axial Refinement Network for Monocular 3D Object Detection [160.34246529816085]
  Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. We propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it.
  arXiv  Detail & Related papers  (2020-08-31T17:10:48Z)
• Recalibrating 3D ConvNets with Project & Excite [6.11737116137921]
  Convolutional Neural Networks (F-CNNs) achieve state-of-the-art performance for segmentation tasks in computer vision and medical imaging. We extend existing 2D recalibration methods to 3D and propose a generic compress-process-recalibrate pipeline for easy comparison. We demonstrate that PE modules can be easily integrated into 3D F-CNNs, boosting performance up to 0.3 in Dice Score and outperforming 3D extensions of other recalibration blocks.
  arXiv  Detail & Related papers  (2020-02-25T16:07:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.