Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery

• URL: http://arxiv.org/abs/2607.08408v1
• Date: Thu, 09 Jul 2026 12:34:55 GMT
• Title: Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery
• Abstract summary: We propose Track2Map, an online 3D Gaussian splatting pipeline.<n>Track2Map is capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy.<n> Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods.
• Score: 13.747762932103429
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gaussian splatting is the current state-of-the-art for dense, deformable 3D anatomy reconstruction in robot-assisted minimally invasive surgery (RAMIS); however, most pipelines are offline and depend on accurate camera trajectory priors (often from robotic kinematics), limiting applicability when priors are missing or noisy. To address these limitations, we propose Track2Map, an online 3D Gaussian Splatting pipeline that jointly optimizes camera trajectory and 3D deformable scene representation directly from surgical video. Track2Map is therefore capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy, and due to its online nature it effectively works as a Simultaneous Localisation and Mapping (SLAM) method. To stabilize optimization in the presence of tissue motion and ambiguous visual cues, we introduce a track-anchored deformation initialization using dense 2D point tracks. Track statistics are further utilized to disentangle camera motion from scene deformation by detecting static camera periods and reducing drift during incremental mapping. Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods, as well as compared to non-SLAM methods that utilize camera trajectory priors. The code is available at https://track2map.github.io/.

Related papers

• BSGS: Bi-stage 3D Gaussian Splatting for Camera Motion Deblurring [32.5099324617965]
  Bi-Stage 3D Gaussian Splatting is a novel framework to reconstruct 3D scenes from motion-blurred images.<n>We propose a subframe gradient aggregation strategy to optimize both stages.<n>Experiments verify the effectiveness of our proposed deblurring method and show its superiority over the state of the arts.
  arXiv  Detail & Related papers  (2025-10-14T13:26:56Z)
• GaVS: 3D-Grounded Video Stabilization via Temporally-Consistent Local Reconstruction and Rendering [54.489285024494855]
  Video stabilization is pivotal for video processing, as it removes unwanted shakiness while preserving the original user motion intent.<n>Existing approaches, depending on the domain they operate, suffer from several issues that degrade the user experience.<n>We introduce textbfGaVS, a novel 3D-grounded approach that reformulates video stabilization as a temporally-consistent local reconstruction and rendering' paradigm.
  arXiv  Detail & Related papers  (2025-06-30T15:24:27Z)
• MBA-SLAM: Motion Blur Aware Gaussian Splatting SLAM [15.752529196306648]
  We propose a dense visual deblur SLAM pipeline (i.e. MBA-SLAM) to handle severe motion-blurred inputs and enhance image deblurring.<n>Our approach integrates an efficient motion blur-aware tracker with either neural fields or Gaussian Splatting based mapper.<n>In experiments, we demonstrate that MBA-SLAM surpasses previous state-of-the-art methods in both camera localization and map reconstruction.
  arXiv  Detail & Related papers  (2024-11-13T01:38:06Z)
• IncEventGS: Pose-Free Gaussian Splatting from a Single Event Camera [6.879406129086464]
  IncEventGS is an incremental 3D Gaussian splatting reconstruction algorithm with a single event camera.<n>We exploit the tracking and mapping paradigm of conventional SLAM pipelines for IncEventGS.
  arXiv  Detail & Related papers  (2024-10-10T16:54:23Z)
• MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting [56.785233997533794]
  We propose a novel deformable 3D Gaussian splatting framework called MotionGS. MotionGS explores explicit motion priors to guide the deformation of 3D Gaussians. Experiments in the monocular dynamic scenes validate that MotionGS surpasses state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-10T08:19:47Z)
• DynOMo: Online Point Tracking by Dynamic Online Monocular Gaussian Reconstruction [65.46359561104867]
  We target the challenge of online 2D and 3D point tracking from unposed monocular camera input.<n>We leverage 3D Gaussian splatting to reconstruct dynamic scenes in an online fashion.<n>We aim to inspire the community to advance online point tracking and reconstruction, expanding the applicability to diverse real-world scenarios.
  arXiv  Detail & Related papers  (2024-09-03T17:58:03Z)
• Fast Encoder-Based 3D from Casual Videos via Point Track Processing [22.563073026889324]
  We present TracksTo4D, a learning-based approach that enables inferring 3D structure and camera positions from dynamic content originating from casual videos. TracksTo4D is trained in an unsupervised way on a dataset of casual videos. Experiments show that TracksTo4D can reconstruct a temporal point cloud and camera positions of the underlying video with accuracy comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-04-10T15:37:00Z)
• MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements [59.70107451308687]
  We show for the first time that using 3D Gaussians for map representation with unposed camera images and inertial measurements can enable accurate SLAM. Our method, MM3DGS, addresses the limitations of prior rendering by enabling faster scale awareness, and improved trajectory tracking. We also release a multi-modal dataset, UT-MM, collected from a mobile robot equipped with a camera and an inertial measurement unit.
  arXiv  Detail & Related papers  (2024-04-01T04:57:41Z)
• Cinematic Behavior Transfer via NeRF-based Differentiable Filming [63.1622492808519]
  Existing SLAM methods face limitations in dynamic scenes and human pose estimation often focuses on 2D projections. We first introduce a reverse filming behavior estimation technique. We then introduce a cinematic transfer pipeline that is able to transfer various shot types to a new 2D video or a 3D virtual environment.
  arXiv  Detail & Related papers  (2023-11-29T15:56:58Z)
• Minimum Latency Deep Online Video Stabilization [77.68990069996939]
  We present a novel camera path optimization framework for the task of online video stabilization. In this work, we adopt recent off-the-shelf high-quality deep motion models for motion estimation to recover the camera trajectory. Our approach significantly outperforms state-of-the-art online methods both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-12-05T07:37:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.