ToF-Splatting: Dense SLAM using Sparse Time-of-Flight Depth and Multi-Frame Integration

• URL: http://arxiv.org/abs/2504.16545v1
• Date: Wed, 23 Apr 2025 09:19:43 GMT
• Title: ToF-Splatting: Dense SLAM using Sparse Time-of-Flight Depth and Multi-Frame Integration
• Abstract summary: We propose ToF-Splatting, the first 3D Gaussian Splatting-based SLAM pipeline tailored for using effectively very sparse ToF input data.<n>Our approach improves upon the state of the art by introducing a multi-frame integration module, which produces dense depth maps by merging cues from extremely sparse ToF depth, monocular color, and multi-view geometry.
• Score: 40.16200204154956
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time-of-Flight (ToF) sensors provide efficient active depth sensing at relatively low power budgets; among such designs, only very sparse measurements from low-resolution sensors are considered to meet the increasingly limited power constraints of mobile and AR/VR devices. However, such extreme sparsity levels limit the seamless usage of ToF depth in SLAM. In this work, we propose ToF-Splatting, the first 3D Gaussian Splatting-based SLAM pipeline tailored for using effectively very sparse ToF input data. Our approach improves upon the state of the art by introducing a multi-frame integration module, which produces dense depth maps by merging cues from extremely sparse ToF depth, monocular color, and multi-view geometry. Extensive experiments on both synthetic and real sparse ToF datasets demonstrate the viability of our approach, as it achieves state-of-the-art tracking and mapping performances on reference datasets.

Related papers

• Dense Metric Depth Completion from Sparse Direct Time-of-Flight Sensors [31.4906223000791]
  Direct Time-of-Flight (dToF) sensors provide highly accurate metric depth.<n>They are unsuitable for VR/XR, robotics, and 3D perception tasks that require dense metric depth.<n>We present a generalizable framework for dense metric depth completion from sparse dToF measurements.
  arXiv  Detail & Related papers  (2026-08-05T12:02:09Z)
• RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow [51.43025173196566]
  Monocular scene flow estimation aims to recover dense 3D motion from image sequences.<n>We propose RAFT-MSF++, a self-supervised multi-frame framework that fuses temporal features to jointly estimate depth and scene flow.<n>Experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions.
  arXiv  Detail & Related papers  (2026-04-21T11:32:49Z)
• ToFormer: Towards Large-scale Scenario Depth Completion for Lightweight ToF Camera [15.365848097977135]
  Time-of-Flight (ToF) cameras possess compact design and high measurement precision to be applied to various robot tasks.<n> Depth completion has emerged as a potential solution to expand the sensing range of ToF cameras.<n>We propose a full-stack framework that enables depth completion in large-scale scenarios for short-range ToF cameras.
  arXiv  Detail & Related papers  (2026-03-21T06:01:32Z)
• S2ML: Spatio-Spectral Mutual Learning for Depth Completion [56.26679539288063]
  Raw depth images captured by RGB-D cameras often suffer from incomplete depth values due to weak reflections, boundary shadows, and artifacts.<n>Existing methods address this problem through depth completion in the image domain, but they overlook the physical characteristics of raw depth images.<n>We propose a Spatio-Spectral Mutual Learning framework (S2ML) to harmonize the advantages of both spatial and frequency domains for depth completion.
  arXiv  Detail & Related papers  (2025-11-08T15:01:55Z)
• Pseudo Depth Meets Gaussian: A Feed-forward RGB SLAM Baseline [64.42938561167402]
  We propose an online 3D reconstruction method using 3D Gaussian-based SLAM, combined with a feed-forward recurrent prediction module.<n>This approach replaces slow test-time optimization with fast network inference, significantly improving tracking speed.<n>Our method achieves performance on par with the state-of-the-art SplaTAM, while reducing tracking time by more than 90%.
  arXiv  Detail & Related papers  (2025-08-06T16:16:58Z)
• Towards High-Precision Depth Sensing via Monocular-Aided iToF and RGB Integration [11.077863605272668]
  We present a novel iToF-RGB fusion framework designed to address the inherent limitations of indirect Time-of-Flight (iToF) depth sensing.<n>The proposed method first reprojects the narrow-FoV iToF depth map onto the wide-FoV RGB coordinate system.<n>A dual-encoder fusion network is then employed to jointly extract complementary features from the reprojected iToF depth and RGB image.<n>By integrating cross-modal structural cues and depth consistency constraints, our approach achieves enhanced depth accuracy, improved edge sharpness, and seamless FoV expansion.
  arXiv  Detail & Related papers  (2025-08-03T13:48:00Z)
• Self-Supervised Enhancement for Depth from a Lightweight ToF Sensor with Monocular Images [7.317782580649895]
  We propose a self-supervised learning framework, SelfToF, which generates detailed and scale-aware depth maps.<n>Our proposed method is both efficient and effective, as verified by extensive experiments on the NYU and ScanNet datasets.
  arXiv  Detail & Related papers  (2025-06-16T12:57:58Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [63.87313550399871]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose Self-supervised Transfer (PST) and FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• SVDC: Consistent Direct Time-of-Flight Video Depth Completion with Frequency Selective Fusion [8.23422182642083]
  Lightweight direct Time-of-Flight (dToF) sensors are ideal for 3D sensing on mobile devices.<n>In this paper, we propose a novel video depth completion method, called SVDC, by fusing the sparse dToF data with the corresponding RGB guidance.<n>Our method employs a multi-frame fusion scheme to mitigate the spatial ambiguity resulting from the sparse dToF imaging.
  arXiv  Detail & Related papers  (2025-03-03T07:32:25Z)
• SLAIM: Robust Dense Neural SLAM for Online Tracking and Mapping [15.63276368052395]
  We propose a novel coarse-to-fine tracking model tailored for Neural Radiance Field SLAM (NeRF-SLAM) Existing NeRF-SLAM systems consistently exhibit inferior tracking performance compared to traditional SLAM algorithms. We implement both local and global bundle-adjustment to produce a robust (coarse-to-fine) and accurate (KL regularizer) SLAM solution.
  arXiv  Detail & Related papers  (2024-04-17T14:23:28Z)
• Ternary-Type Opacity and Hybrid Odometry for RGB NeRF-SLAM [58.736472371951955]
  We introduce a ternary-type opacity (TT) model, which categorizes points on a ray intersecting a surface into three regions: before, on, and behind the surface. This enables a more accurate rendering of depth, subsequently improving the performance of image warping techniques. Our integrated approach of TT and HO achieves state-of-the-art performance on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-12-20T18:03:17Z)
• Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor [58.305341034419136]
  We present the first dense SLAM system with a monocular camera and a light-weight ToF sensor. We propose a multi-modal implicit scene representation that supports rendering both the signals from the RGB camera and light-weight ToF sensor. Experiments demonstrate that our system well exploits the signals of light-weight ToF sensors and achieves competitive results.
  arXiv  Detail & Related papers  (2023-08-28T07:56:13Z)
• Consistent Direct Time-of-Flight Video Depth Super-Resolution [9.173767380836852]
  Direct time-of-flight (dToF) sensors are promising for next-generation on-device 3D sensing. We propose the first multi-frame fusion scheme to mitigate the spatial ambiguity resulting from the low-resolution dToF imaging. We introduce DyDToF, the first synthetic RGB-dToF video dataset that features dynamic objects and a realistic dToF simulator.
  arXiv  Detail & Related papers  (2022-11-16T04:16:20Z)
• CLONeR: Camera-Lidar Fusion for Occupancy Grid-aided Neural Representations [77.90883737693325]
  This paper proposes CLONeR, which significantly improves upon NeRF by allowing it to model large outdoor driving scenes observed from sparse input sensor views. This is achieved by decoupling occupancy and color learning within the NeRF framework into separate Multi-Layer Perceptrons (MLPs) trained using LiDAR and camera data, respectively. In addition, this paper proposes a novel method to build differentiable 3D Occupancy Grid Maps (OGM) alongside the NeRF model, and leverage this occupancy grid for improved sampling of points along a ray for rendering in metric space.
  arXiv  Detail & Related papers  (2022-09-02T17:44:50Z)
• Learning an Efficient Multimodal Depth Completion Model [11.740546882538142]
  RGB image-guided sparse depth completion has attracted extensive attention recently, but still faces some problems. The proposed method can outperform some state-of-the-art methods with a lightweight architecture. The method also wins the championship in the MIPI2022 RGB+TOF depth completion challenge.
  arXiv  Detail & Related papers  (2022-08-23T07:03:14Z)
• A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection [89.88222217065858]
  We design a single stream network to use the depth map to guide early fusion and middle fusion between RGB and depth. This model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a $384 times 384$ image.
  arXiv  Detail & Related papers  (2020-07-14T04:40:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.