ToFormer: Towards Large-scale Scenario Depth Completion for Lightweight ToF Camera

• URL: http://arxiv.org/abs/2603.20669v1
• Date: Sat, 21 Mar 2026 06:01:32 GMT
• Title: ToFormer: Towards Large-scale Scenario Depth Completion for Lightweight ToF Camera
• Abstract summary: Time-of-Flight (ToF) cameras possess compact design and high measurement precision to be applied to various robot tasks.<n> Depth completion has emerged as a potential solution to expand the sensing range of ToF cameras.<n>We propose a full-stack framework that enables depth completion in large-scale scenarios for short-range ToF cameras.
• Score: 15.365848097977135
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time-of-Flight (ToF) cameras possess compact design and high measurement precision to be applied to various robot tasks. However, their limited sensing range restricts deployment in large-scale scenarios. Depth completion has emerged as a potential solution to expand the sensing range of ToF cameras, but existing research lacks dedicated datasets and struggles to generalize to ToF measurements. In this paper, we propose a full-stack framework that enables depth completion in large-scale scenarios for short-range ToF cameras. First, we construct a multi-sensor platform with a reconstruction-based pipeline to collect real-world ToF samples with dense large-scale ground truth, yielding the first LArge-ScalE scenaRio ToF depth completion dataset (LASER-ToF). Second, we propose a sensor-aware depth completion network that incorporates a novel 3D branch with a 3D-2D Joint Propagation Pooling (JPP) module and Multimodal Cross-Covariance Attention (MXCA), enabling effective modeling of long-range relationships and efficient 3D-2D fusion under non-uniform ToF depth sparsity. Moreover, our network can utilize the sparse point cloud from visual SLAM as a supplement to ToF depth to further improve prediction accuracy. Experiments show that our method achieves an 8.6% lower mean absolute error than the second-best method, while maintaining lightweight design to support onboard deployment. Finally, to verify the system's applicability on real robots, we deploy proposed method on a quadrotor at a 10Hz runtime, enabling reliable large-scale mapping and long-range planning in challenging environments for short-range ToF cameras.

Related papers

• Dense Metric Depth Completion from Sparse Direct Time-of-Flight Sensors [31.4906223000791]
  Direct Time-of-Flight (dToF) sensors provide highly accurate metric depth.<n>They are unsuitable for VR/XR, robotics, and 3D perception tasks that require dense metric depth.<n>We present a generalizable framework for dense metric depth completion from sparse dToF measurements.
  arXiv  Detail & Related papers  (2026-08-05T12:02:09Z)
• FreqPDE: Rethinking Positional Depth Embedding for Multi-View 3D Object Detection Transformers [91.59069344768858]
  We introduce Frequency-aware Positional Depth Embedding (FreqPDE) to equip 2D image features with spatial information for 3D detection transformer decoder.<n>FreqPDE combines the 2D image features and 3D position embeddings to generate 3D depth-aware features for query decoding.
  arXiv  Detail & Related papers  (2025-10-17T07:36:54Z)
• Self-Supervised Enhancement for Depth from a Lightweight ToF Sensor with Monocular Images [7.317782580649895]
  We propose a self-supervised learning framework, SelfToF, which generates detailed and scale-aware depth maps.<n>Our proposed method is both efficient and effective, as verified by extensive experiments on the NYU and ScanNet datasets.
  arXiv  Detail & Related papers  (2025-06-16T12:57:58Z)
• ToF-Splatting: Dense SLAM using Sparse Time-of-Flight Depth and Multi-Frame Integration [40.16200204154956]
  We propose ToF-Splatting, the first 3D Gaussian Splatting-based SLAM pipeline tailored for using effectively very sparse ToF input data.<n>Our approach improves upon the state of the art by introducing a multi-frame integration module, which produces dense depth maps by merging cues from extremely sparse ToF depth, monocular color, and multi-view geometry.
  arXiv  Detail & Related papers  (2025-04-23T09:19:43Z)
• DoF-Gaussian: Controllable Depth-of-Field for 3D Gaussian Splatting [52.52398576505268]
  We introduce DoF-Gaussian, a controllable depth-of-field method for 3D-GS.<n>We develop a lens-based imaging model based on geometric optics principles to control DoF effects.<n>Our framework is customizable and supports various interactive applications.
  arXiv  Detail & Related papers  (2025-03-02T05:57:57Z)
• Consistent Direct Time-of-Flight Video Depth Super-Resolution [9.173767380836852]
  Direct time-of-flight (dToF) sensors are promising for next-generation on-device 3D sensing. We propose the first multi-frame fusion scheme to mitigate the spatial ambiguity resulting from the low-resolution dToF imaging. We introduce DyDToF, the first synthetic RGB-dToF video dataset that features dynamic objects and a realistic dToF simulator.
  arXiv  Detail & Related papers  (2022-11-16T04:16:20Z)
• DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image [39.389538555506256]
  We propose DELTAR, a novel method to empower light-weight ToF sensors with the capability of measuring high resolution and accurate depth. As the core of DELTAR, a feature extractor customized for depth distribution and an attention-based neural architecture is proposed to fuse the information from the color and ToF domain efficiently. Experiments show that our method produces more accurate depth than existing frameworks designed for depth completion and depth super-resolution and achieves on par performance with a commodity-level RGB-D sensor.
  arXiv  Detail & Related papers  (2022-09-27T13:11:37Z)
• FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction [73.85709132666626]
  Existing representations, such as parametric models, voxel grids, meshes and implicit neural representations, have difficulties achieving high-quality results and real-time speed at the same time. We propose Fourier Occupancy Field (FOF), a novel powerful, efficient and flexible 3D representation, for monocular real-time and accurate human reconstruction. A FOF can be stored as a multi-channel image, which is compatible with 2D convolutional neural networks and can bridge the gap between 3D and 2D images.
  arXiv  Detail & Related papers  (2022-06-05T14:45:02Z)
• Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments [56.306567220448684]
  We propose a new learning based end-to-end depth prediction network which takes noisy raw I-ToF signals as well as an RGB image. We show more than 40% RMSE improvement on the final depth map compared to the baseline approach.
  arXiv  Detail & Related papers  (2021-12-07T15:04:14Z)
• A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection [89.88222217065858]
  We design a single stream network to use the depth map to guide early fusion and middle fusion between RGB and depth. This model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a $384 times 384$ image.
  arXiv  Detail & Related papers  (2020-07-14T04:40:14Z)
• Video Depth Estimation by Fusing Flow-to-Depth Proposals [65.24533384679657]
  We present an approach with a differentiable flow-to-depth layer for video depth estimation. The model consists of a flow-to-depth layer, a camera pose refinement module, and a depth fusion network. Our approach outperforms state-of-the-art depth estimation methods, and has reasonable cross dataset generalization capability.
  arXiv  Detail & Related papers  (2019-12-30T10:45:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.