EfficientPENet: Real-Time Depth Completion from Sparse LiDAR via Lightweight Multi-Modal Fusion

• URL: http://arxiv.org/abs/2604.18790v1
• Date: Mon, 20 Apr 2026 19:56:04 GMT
• Title: EfficientPENet: Real-Time Depth Completion from Sparse LiDAR via Lightweight Multi-Modal Fusion
• Abstract summary: Existing methods achieve high accuracy on standard benchmarks but rely on heavy backbone architectures that preclude real-time deployment on embedded hardware.<n>We present EfficientPENet, a two-branch depth completion network that replaces the conventional ResNet encoder with a modernized ConvNeXt backbone.<n>On the KITTI depth completion benchmark, EfficientPENet achieves an RMSE of 631.94 mm with 36.24M parameters and a latency of 20.51 ms, operating at 48.76 FPS.
• Score: 1.9903316442426757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Depth completion from sparse LiDAR measurements and corresponding RGB images is a prerequisite for accurate 3D perception in robotic systems. Existing methods achieve high accuracy on standard benchmarks but rely on heavy backbone architectures that preclude real-time deployment on embedded hardware. We present EfficientPENet, a two-branch depth completion network that replaces the conventional ResNet encoder with a modernized ConvNeXt backbone, introduces sparsity-invariant convolutions for the depth stream, and refines predictions through a Convolutional Spatial Propagation Network (CSPN). The RGB branch leverages ImageNet-pretrained ConvNeXt blocks with Layer Normalization, 7x7 depthwise convolutions, and stochastic depth regularization. Features from both branches are merged via late fusion and decoded through a multi-scale deep supervision strategy. We further introduce a position-aware test-time augmentation scheme that corrects coordinate tensors during horizontal flipping, yielding consistent error reduction at inference. On the KITTI depth completion benchmark, EfficientPENet achieves an RMSE of 631.94 mm with 36.24M parameters and a latency of 20.51 ms, operating at 48.76 FPS. This represents a 3.7 times reduction in parameters and a 23 times speedup relative to BP-Net, while maintaining competitive accuracy. These results establish EfficientPENet as a practical solution for real-time depth completion on resource-constrained edge platforms such as the NVIDIA Jetson.

Related papers

• Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing [55.13802890769086]
  We present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing.<n>To achieve profound modal synergy, Evita embeds a suite of intrinsic co-learning modules directly into every encoder layer.<n>Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.
  arXiv  Detail & Related papers  (2026-07-10T06:52:09Z)
• Pip-Stereo: Progressive Iterations Pruner for Iterative Optimization based Stereo Matching [2.226667542457085]
  We analyze iterative refinement and reveal that disparity updates are spatially sparse and temporally redundant.<n>We propose a collaborative monocular prior transfer framework that implicitly embeds depth priors without requiring a dedicated monocular encoder.<n>Third, we develop FlashGRU, a hardware-aware RNN operator leveraging structured sparsity and I/O-conscious design.
  arXiv  Detail & Related papers  (2026-02-24T02:51:37Z)
• Pseudo Depth Meets Gaussian: A Feed-forward RGB SLAM Baseline [64.42938561167402]
  We propose an online 3D reconstruction method using 3D Gaussian-based SLAM, combined with a feed-forward recurrent prediction module.<n>This approach replaces slow test-time optimization with fast network inference, significantly improving tracking speed.<n>Our method achieves performance on par with the state-of-the-art SplaTAM, while reducing tracking time by more than 90%.
  arXiv  Detail & Related papers  (2025-08-06T16:16:58Z)
• Speedy MASt3R [68.47052557089631]
  MASt3R redefines image matching as a 3D task by leveraging DUSt3R and introducing a fast reciprocal matching scheme.<n>Fast MASt3R achieves a 54% reduction in inference time (198 ms to 91 ms per image pair) without sacrificing accuracy.<n>This advancement enables real-time 3D understanding, benefiting applications like mixed reality navigation and large-scale 3D scene reconstruction.
  arXiv  Detail & Related papers  (2025-03-13T03:56:22Z)
• Gated Cross-Attention Network for Depth Completion [11.154059839763738]
  Fusion of color and depth features is the current critical challenge in the field of depth estimation. We design an efficient Gated Cross-Attention Network that propagates confidence via a gating mechanism. We use the Ray Tune mechanism with the AsyncHyperBandScheduler scheduler and the HyperOptSearch algorithm to automatically search for the optimal number of module iterations.
  arXiv  Detail & Related papers  (2023-09-28T09:54:10Z)
• Real-time Monocular Depth Estimation on Embedded Systems [32.40848141360501]
  Two efficient RT-MonoDepth and RT-MonoDepth-S architectures are proposed. RT-MonoDepth and RT-MonoDepth-S achieve frame rates of 18.4&30.5 FPS on NVIDIA Jetson Nano and 253.0&364.1 FPS on Jetson AGX Orin.
  arXiv  Detail & Related papers  (2023-08-21T08:59:59Z)
• Rethinking Lightweight Salient Object Detection via Network Depth-Width Tradeoff [26.566339984225756]
  Existing salient object detection methods often adopt deeper and wider networks for better performance. We propose a novel trilateral decoder framework by decoupling the U-shape structure into three complementary branches. We show that our method achieves better efficiency-accuracy balance across five benchmarks.
  arXiv  Detail & Related papers  (2023-01-17T03:43:25Z)
• CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth [83.77839773394106]
  We present a lightweight, tightly-coupled deep depth network and visual-inertial odometry system. We provide the network with previously marginalized sparse features from VIO to increase the accuracy of initial depth prediction. We show that it can run in real-time with single-thread execution while utilizing GPU acceleration only for the network and code Jacobian.
  arXiv  Detail & Related papers  (2020-12-18T09:42:54Z)
• Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection [63.18846475183332]
  We aim to develop an efficient and compact deep network for RGB-D salient object detection. We propose a progressively guided alternate refinement network to refine it. Our model outperforms existing state-of-the-art approaches by a large margin.
  arXiv  Detail & Related papers  (2020-08-17T02:55:06Z)
• A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection [89.88222217065858]
  We design a single stream network to use the depth map to guide early fusion and middle fusion between RGB and depth. This model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a $384 times 384$ image.
  arXiv  Detail & Related papers  (2020-07-14T04:40:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.