Dense Metric Depth Completion from Sparse Direct Time-of-Flight Sensors

• URL: http://arxiv.org/abs/2608.04737v1
• Date: Wed, 05 Aug 2026 12:02:09 GMT
• Title: Dense Metric Depth Completion from Sparse Direct Time-of-Flight Sensors
• Abstract summary: Direct Time-of-Flight (dToF) sensors provide highly accurate metric depth.<n>They are unsuitable for VR/XR, robotics, and 3D perception tasks that require dense metric depth.<n>We present a generalizable framework for dense metric depth completion from sparse dToF measurements.
• Score: 31.4906223000791
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Direct Time-of-Flight (dToF) sensors provide highly accurate metric depth and are more robust than indirect ToF systems in challenging real-world conditions. However, their high manufacturing cost and limited photodiode array size produce depth maps that are extremely sparse, low-resolution, and noisy, making them unsuitable for VR/XR, robotics, and 3D perception tasks that require dense metric depth. Existing monocular and depth completion methods struggle to handle the unique sampling patterns and hardware artifacts of dToF devices, and their performance often deteriorates significantly under severe sparsity or noise. We present a generalizable framework for dense metric depth completion from sparse dToF measurements, capable of operating across diverse sensor types, sparsity levels, and noise conditions. Our model employs a depth-guided dual-branch Vision Transformer encoder that processes RGB images and sparse dToF measurements separately, while a masked joint attention module allows depth tokens to reliably guide image features without being overwritten by them. A lightweight decoder reconstructs dense metric depth efficiently, without diffusion-based or refinement-heavy post-processing. To address the scarcity of paired training data, we introduce a comprehensive dToF simulation pipeline that reproduces the characteristics of flash, sub-VGA flash, and rotating sensors, including hardware-induced degradation, irregular sparsity, and realistic noise distributions. Trained entirely on synthetic data, our model achieves strong zero-shot generalization across 6 datasets and 3 real dToF devices, outperforming state-of-the-art approaches in both accuracy and computational efficiency. This establishes a robust and practical solution for dense metric depth completion from sparse direct ToF sensors. Our code and models are open-sourced. See https://vclab.kaist.ac.kr/cvpr2026p3.

Related papers

• UDPNet: Unleashing Depth-based Priors for Robust Image Dehazing [77.10640210751981]
  UDPNet is a general framework that leverages depth-based priors from a large-scale pretrained depth estimation model DepthAnything V2.<n>Our proposed solution establishes a new benchmark for depth-aware dehazing across various scenarios.
  arXiv  Detail & Related papers  (2026-01-11T13:29:02Z)
• Geometry-Aware Sparse Depth Sampling for High-Fidelity RGB-D Depth Completion in Robotic Systems [0.20999222360659608]
  RGB-D and stereo vision sensors are widely used for industrial robotic systems that perform manipulation, inspection, and navigation tasks.<n>Current depth completion methods produce noisy, incomplete, or biased depth maps due to sensor limitations and environmental conditions.<n>We propose a normal-guided sparse depth sampling strategy that leverages PCA-based surface normal estimation on the RGB-D point cloud to compute a per-pixel depth reliability measure.<n> Experiments show that our geometry-aware sparse depth improves accuracy, reduces artifacts near edges and discontinuities, and produces more realistic training conditions.
  arXiv  Detail & Related papers  (2025-12-09T04:14:05Z)
• S2ML: Spatio-Spectral Mutual Learning for Depth Completion [56.26679539288063]
  Raw depth images captured by RGB-D cameras often suffer from incomplete depth values due to weak reflections, boundary shadows, and artifacts.<n>Existing methods address this problem through depth completion in the image domain, but they overlook the physical characteristics of raw depth images.<n>We propose a Spatio-Spectral Mutual Learning framework (S2ML) to harmonize the advantages of both spatial and frequency domains for depth completion.
  arXiv  Detail & Related papers  (2025-11-08T15:01:55Z)
• FreqPDE: Rethinking Positional Depth Embedding for Multi-View 3D Object Detection Transformers [91.59069344768858]
  We introduce Frequency-aware Positional Depth Embedding (FreqPDE) to equip 2D image features with spatial information for 3D detection transformer decoder.<n>FreqPDE combines the 2D image features and 3D position embeddings to generate 3D depth-aware features for query decoding.
  arXiv  Detail & Related papers  (2025-10-17T07:36:54Z)
• DEPTHOR++: Robust Depth Enhancement from a Real-World Lightweight dToF and RGB Guidance [14.818201604060144]
  DEPTHOR++ is a practical and novel depth completion framework.<n>It enhances robustness to noisy dToF inputs from three key aspects.<n>On the ZJU-L5 dataset and real-world samples, our training strategy significantly boosts existing depth completion models.
  arXiv  Detail & Related papers  (2025-09-30T16:41:11Z)
• DGFusion: Depth-Guided Sensor Fusion for Robust Semantic Perception [57.77346327566903]
  State-of-the-art sensor fusion approaches to semantic perception often treat sensor data uniformly across the spatial extent of the input.<n>We propose a novel depth-guided multimodal fusion method that upgrades condition-aware fusion by integrating depth information.<n>Our method achieves state-of-the-art panoptic and semantic segmentation performance on the challenging MUSES and DELIVER datasets.
  arXiv  Detail & Related papers  (2025-09-11T20:03:00Z)
• Self-Supervised Enhancement for Depth from a Lightweight ToF Sensor with Monocular Images [7.317782580649895]
  We propose a self-supervised learning framework, SelfToF, which generates detailed and scale-aware depth maps.<n>Our proposed method is both efficient and effective, as verified by extensive experiments on the NYU and ScanNet datasets.
  arXiv  Detail & Related papers  (2025-06-16T12:57:58Z)
• Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor [58.305341034419136]
  We present the first dense SLAM system with a monocular camera and a light-weight ToF sensor. We propose a multi-modal implicit scene representation that supports rendering both the signals from the RGB camera and light-weight ToF sensor. Experiments demonstrate that our system well exploits the signals of light-weight ToF sensors and achieves competitive results.
  arXiv  Detail & Related papers  (2023-08-28T07:56:13Z)
• Consistent Direct Time-of-Flight Video Depth Super-Resolution [9.173767380836852]
  Direct time-of-flight (dToF) sensors are promising for next-generation on-device 3D sensing. We propose the first multi-frame fusion scheme to mitigate the spatial ambiguity resulting from the low-resolution dToF imaging. We introduce DyDToF, the first synthetic RGB-dToF video dataset that features dynamic objects and a realistic dToF simulator.
  arXiv  Detail & Related papers  (2022-11-16T04:16:20Z)
• DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image [39.389538555506256]
  We propose DELTAR, a novel method to empower light-weight ToF sensors with the capability of measuring high resolution and accurate depth. As the core of DELTAR, a feature extractor customized for depth distribution and an attention-based neural architecture is proposed to fuse the information from the color and ToF domain efficiently. Experiments show that our method produces more accurate depth than existing frameworks designed for depth completion and depth super-resolution and achieves on par performance with a commodity-level RGB-D sensor.
  arXiv  Detail & Related papers  (2022-09-27T13:11:37Z)
• Joint Learning of Salient Object Detection, Depth Estimation and Contour Extraction [91.43066633305662]
  We propose a novel multi-task and multi-modal filtered transformer (MMFT) network for RGB-D salient object detection (SOD) Specifically, we unify three complementary tasks: depth estimation, salient object detection and contour estimation. The multi-task mechanism promotes the model to learn the task-aware features from the auxiliary tasks. Experiments show that it not only significantly surpasses the depth-based RGB-D SOD methods on multiple datasets, but also precisely predicts a high-quality depth map and salient contour at the same time.
  arXiv  Detail & Related papers  (2022-03-09T17:20:18Z)
• A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection [89.88222217065858]
  We design a single stream network to use the depth map to guide early fusion and middle fusion between RGB and depth. This model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a $384 times 384$ image.
  arXiv  Detail & Related papers  (2020-07-14T04:40:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.