RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction

• URL: http://arxiv.org/abs/2603.20588v1
• Date: Sat, 21 Mar 2026 01:04:31 GMT
• Title: RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction
• Abstract summary: Streaming feed-forward 3D reconstruction enables real-time joint estimation of scene geometry and camera poses from RGB images.<n>We propose RayMap3R, a training-free streaming framework for dynamic scene reconstruction.
• Score: 22.98233376526841
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Streaming feed-forward 3D reconstruction enables real-time joint estimation of scene geometry and camera poses from RGB images. However, without explicit dynamic reasoning, streaming models can be affected by moving objects, causing artifacts and drift. In this work, we propose RayMap3R, a training-free streaming framework for dynamic scene reconstruction. We observe that RayMap-based predictions exhibit a static-scene bias, providing an internal cue for dynamic identification. Based on this observation, we construct a dual-branch inference scheme that identifies dynamic regions by contrasting RayMap and image predictions, suppressing their interference during memory updates. We further introduce reset metric alignment and state-aware smoothing to preserve metric consistency and stabilize predicted trajectories. Our method achieves state-of-the-art performance among streaming approaches on dynamic scene reconstruction across multiple benchmarks.

Related papers

• Raymoval: Raycasting-based Dynamic Object Removal for Static 3D Mapping [8.410344664123071]
  We propose a raycasting-based module for dynamic object removal in static 3D mapping.<n>Each scan is projected onto an azimuth-elevation grid, and for every viewing direction we compare the bin-wise minimum range with the map's first-hit distance computed by raycasting.<n>Finally, a spatial consistency validation step refines labels, producing static maps with lower residual dynamics and reduced over-removal.
  arXiv  Detail & Related papers  (2026-05-09T13:15:59Z)
• Flow4R: Unifying 4D Reconstruction and Tracking with Scene Flow [61.297800738187355]
  Flow4R predicts a minimal per-pixel property set-3D point position, scene flow, pose weight, and confidence-from two-view inputs using a Vision Transformer.<n> trained jointly on static and dynamic datasets, Flow4R achieves state-of-the-art performance on 4D reconstruction and tracking tasks.
  arXiv  Detail & Related papers  (2026-02-15T06:58:08Z)
• St4RTrack: Simultaneous 4D Reconstruction and Tracking in the World [106.91539872943864]
  St4RTrack is a framework that simultaneously reconstructs and tracks dynamic video content in a world coordinate frame from RGB inputs.<n>We predict both pointmaps at the same moment, in the same world, capturing both static and dynamic scene geometry.<n>We establish a new extensive benchmark for world-frame reconstruction and tracking, demonstrating the effectiveness and efficiency of our unified, data-driven framework.
  arXiv  Detail & Related papers  (2025-04-17T17:55:58Z)
• D$^2$USt3R: Enhancing 3D Reconstruction for Dynamic Scenes [54.886845755635754]
  This work addresses the task of 3D reconstruction in dynamic scenes, where object motions frequently degrade the quality of previous 3D pointmap regression methods.<n>By explicitly incorporating both spatial and temporal aspects, our approach successfully encapsulates 3D dense correspondence to the proposed pointmaps.
  arXiv  Detail & Related papers  (2025-04-08T17:59:50Z)
• POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstruction [53.19968902152528]
  We present POMATO, a unified framework for dynamic 3D reconstruction by marrying pointmap matching with temporal motion.<n>Specifically, our method learns an explicit matching relationship by mapping RGB pixels from both dynamic and static regions across different views to 3D pointmaps.<n>We show the effectiveness of the proposed pointmap matching and temporal fusion paradigm by demonstrating the remarkable performance across multiple downstream tasks.
  arXiv  Detail & Related papers  (2025-04-08T05:33:13Z)
• Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstruction [56.32589034046427]
  We introduce Dynamic Point Maps (DPM), extending standard point maps to support 4D tasks such as motion segmentation, scene flow estimation, 3D object tracking, and 2D correspondence.<n>We train a DPM predictor on a mixture of synthetic and real data and evaluate it across diverse benchmarks for video depth prediction, dynamic point cloud reconstruction, 3D scene flow and object pose tracking, achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-20T16:41:50Z)
• Driv3R: Learning Dense 4D Reconstruction for Autonomous Driving [116.10577967146762]
  We propose Driv3R, a framework that directly regresses per-frame point maps from multi-view image sequences.<n>We employ a 4D flow predictor to identify moving objects within the scene to direct our network focus more on reconstructing these dynamic regions.<n>Driv3R outperforms previous frameworks in 4D dynamic scene reconstruction, achieving 15x faster inference speed.
  arXiv  Detail & Related papers  (2024-12-09T18:58:03Z)
• Dynamic Semantic Occupancy Mapping using 3D Scene Flow and Closed-Form Bayesian Inference [3.0389083199673337]
  We leverage state-of-the-art semantic segmentation and 3D flow estimation using deep learning to provide measurements for map inference. We develop a continuous (i.e., can be queried at arbitrary resolution) Bayesian model that propagates the scene with flow and infers a 3D semantic occupancy map with better performance than its static counterpart.
  arXiv  Detail & Related papers  (2021-08-06T15:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.