Dynamic Spatial Propagation Network for Depth Completion

• URL: http://arxiv.org/abs/2202.09769v1
• Date: Sun, 20 Feb 2022 09:43:17 GMT
• Title: Dynamic Spatial Propagation Network for Depth Completion
• Authors: Yuankai Lin, Tao Cheng, Qi Zhong, Wending Zhou and Hua Yang
• Abstract summary: This paper introduces an efficient model that learns the affinity among neighboring pixels with an attention-based approach. In practice, our method requires less iteration to match the performance of other SPNs and yields better results overall.
• Score: 6.3447233767041356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image-guided depth completion aims to generate dense depth maps with sparse depth measurements and corresponding RGB images. Currently, spatial propagation networks (SPNs) are the most popular affinity-based methods in depth completion, but they still suffer from the representation limitation of the fixed affinity and the over smoothing during iterations. Our solution is to estimate independent affinity matrices in each SPN iteration, but it is over-parameterized and heavy calculation. This paper introduces an efficient model that learns the affinity among neighboring pixels with an attention-based, dynamic approach. Specifically, the Dynamic Spatial Propagation Network (DySPN) we proposed makes use of a non-linear propagation model (NLPM). It decouples the neighborhood into parts regarding to different distances and recursively generates independent attention maps to refine these parts into adaptive affinity matrices. Furthermore, we adopt a diffusion suppression (DS) operation so that the model converges at an early stage to prevent over-smoothing of dense depth. Finally, in order to decrease the computational cost required, we also introduce three variations that reduce the amount of neighbors and attentions needed while still retaining similar accuracy. In practice, our method requires less iteration to match the performance of other SPNs and yields better results overall. DySPN outperforms other state-of-the-art (SoTA) methods on KITTI Depth Completion (DC) evaluation by the time of submission and is able to yield SoTA performance in NYU Depth v2 dataset as well.

Related papers

• Bilateral Propagation Network for Depth Completion [41.163328523175466]
  Depth completion aims to derive a dense depth map from sparse depth measurements with a synchronized color image. Current state-of-the-art (SOTA) methods are predominantly propagation-based, which work as an iterative refinement on the initial estimated dense depth. We present a Bilateral Propagation Network (BP-Net), that propagates depth at the earliest stage to avoid directly convolving on sparse data.
  arXiv  Detail & Related papers  (2024-03-17T16:48:46Z)
• GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs [49.55919802779889]
  We propose a Graph Convolution based Spatial Propagation Network (GraphCSPN) as a general approach for depth completion. In this work, we leverage convolution neural networks as well as graph neural networks in a complementary way for geometric representation learning. Our method achieves the state-of-the-art performance, especially when compared in the case of using only a few propagation steps.
  arXiv  Detail & Related papers  (2022-10-19T17:56:03Z)
• Shallow Network Based on Depthwise Over-Parameterized Convolution for Hyperspectral Image Classification [0.7329200485567825]
  This letter proposes a shallow model for hyperspectral image classification (HSIC) using convolutional neural network (CNN) techniques. The proposed method outperforms other state-of-the-art methods in terms of classification accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2021-12-01T03:10:02Z)
• Manifold Regularized Dynamic Network Pruning [102.24146031250034]
  This paper proposes a new paradigm that dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks. The effectiveness of the proposed method is verified on several benchmarks, which shows better performance in terms of both accuracy and computational cost.
  arXiv  Detail & Related papers  (2021-03-10T03:59:03Z)
• Solving Sparse Linear Inverse Problems in Communication Systems: A Deep Learning Approach With Adaptive Depth [51.40441097625201]
  We propose an end-to-end trainable deep learning architecture for sparse signal recovery problems. The proposed method learns how many layers to execute to emit an output, and the network depth is dynamically adjusted for each task in the inference phase.
  arXiv  Detail & Related papers  (2020-10-29T06:32:53Z)
• Non-Local Spatial Propagation Network for Depth Completion [82.60915972250706]
  We propose a robust and efficient end-to-end non-local spatial propagation network for depth completion. The proposed network takes RGB and sparse depth images as inputs and estimates non-local neighbors and their affinities of each pixel. We show that the proposed algorithm is superior to conventional algorithms in terms of depth completion accuracy and robustness to the mixed-depth problem.
  arXiv  Detail & Related papers  (2020-07-20T12:26:51Z)
• Deformable spatial propagation network for depth completion [2.5306673456895306]
  We propose a deformable spatial propagation network (DSPN) to adaptively generates different receptive field and affinity matrix for each pixel. It allows the network obtain information with much fewer but more relevant pixels for propagation.
  arXiv  Detail & Related papers  (2020-07-08T16:39:50Z)
• Augmented Parallel-Pyramid Net for Attention Guided Pose-Estimation [90.28365183660438]
  This paper proposes an augmented parallel-pyramid net with attention partial module and differentiable auto-data augmentation. We define a new pose search space where the sequences of data augmentations are formulated as a trainable and operational CNN component. Notably, our method achieves the top-1 accuracy on the challenging COCO keypoint benchmark and the state-of-the-art results on the MPII datasets.
  arXiv  Detail & Related papers  (2020-03-17T03:52:17Z)
• Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes [98.65457534223539]
  We propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes. The proposed method achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) with the inference speed of 51.0 fps and 39.3 fps.
  arXiv  Detail & Related papers  (2020-03-11T08:45:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.