Geometric Context Transformer for Streaming 3D Reconstruction

• URL: http://arxiv.org/abs/2604.14141v2
• Date: Thu, 16 Apr 2026 16:44:56 GMT
• Title: Geometric Context Transformer for Streaming 3D Reconstruction
• Abstract summary: LingBot-Map is a feed-forward 3D foundation model for reconstructing scenes from streaming data.<n>A defining aspect of LingBot-Map lies in its carefully designed attention mechanism, which integrates an anchor context, a pose-reference window, and a trajectory memory.<n>This design keeps the streaming state compact while retaining rich geometric context, enabling stable efficient inference at around 20 FPS on 518 x 378 resolution inputs.
• Score: 51.19524805829903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Streaming 3D reconstruction aims to recover 3D information, such as camera poses and point clouds, from a video stream, which necessitates geometric accuracy, temporal consistency, and computational efficiency. Motivated by the principles of Simultaneous Localization and Mapping (SLAM), we introduce LingBot-Map, a feed-forward 3D foundation model for reconstructing scenes from streaming data, built upon a geometric context transformer (GCT) architecture. A defining aspect of LingBot-Map lies in its carefully designed attention mechanism, which integrates an anchor context, a pose-reference window, and a trajectory memory to address coordinate grounding, dense geometric cues, and long-range drift correction, respectively. This design keeps the streaming state compact while retaining rich geometric context, enabling stable efficient inference at around 20 FPS on 518 x 378 resolution inputs over long sequences exceeding 10,000 frames. Extensive evaluations across a variety of benchmarks demonstrate that our approach achieves superior performance compared to both existing streaming and iterative optimization-based approaches.

Related papers

• Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction [10.383154504465232]
  ABot-Recon is a simple streaming model that caches KV features from only the preceding 11 frames.<n>It predicts a point map in the current camera coordinate system together with an adjacent-frame relative pose.<n>To reduce accumulated drift, a lightweight temporal refiner improves relative rotations using recent visual and motion context.
  arXiv  Detail & Related papers  (2026-08-27T15:07:38Z)
• IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer [64.5765465030666]
  We propose IGGT4D, a streaming instance-grounded geometry Transformer for online 4D scene understanding.<n> IGGT4D processes video frames sequentially, reuses historical context through causal spatial-temporal modeling, and incrementally updates a unified representation of camera motion, geometry, and object identity.<n> Experiments on 3D reconstruction, pose estimation, instance spatial tracking, and open-vocabulary segmentation demonstrate that IGGT4D outperforms existing streaming baselines.
  arXiv  Detail & Related papers  (2026-07-21T16:00:01Z)
• 3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation [21.287558280133076]
  We introduce a new paradigm that recasts sequential video depth estimation as an unconstrained multi-view 3D reconstruction problem.<n>Our approach is driven by three constraints: image-level photometric rendering, explicit world-coordinate geometric alignment, and multi-scale temporal gradient consistency.<n>Results show that the proposed approach achieves state-of-the-art spatial accuracy, outperforming the frame-based, video-based depth estimators and the multi-view 3D reconstruction baselines.
  arXiv  Detail & Related papers  (2026-06-14T08:56:45Z)
• Anchor3R: Streaming 3D Reconstruction with Transient Anchors for Long-Horizon Visual Mapping [28.6268034980389]
  Long-horizon online visual mapping is a core capability for robot perception.<n>Recent feed-forward 3D reconstruction models predict poses in a fixed coordinate system tied to the first frame or a persistent scene memory.<n>We propose emphAnchor3R, a streaming 3D reconstruction framework that treats feed-forward reconstruction as current-centric local measurement prediction.
  arXiv  Detail & Related papers  (2026-06-03T16:00:13Z)
• GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth [12.866152238833104]
  Video depth estimation extends monocular prediction into the temporal domain to ensure coherence.<n>Current approaches, which primarily rely on temporal smoothing via Transformers, struggle to maintain strict 3D geometric consistency.<n>We propose GemDepth, a framework built on the insight that an explicit awareness of camera motion and global 3D structure is a prerequisite for 3D consistency.
  arXiv  Detail & Related papers  (2026-05-11T13:11:54Z)
• GGPT: Geometry Grounded Point Transformer [22.64445696362087]
  We introduce the Geometry-Grounded Point Transformer (GGPT), a framework that augments feed-forward reconstruction with reliable sparse geometric guidance.<n>Building on this foundation, we propose a geometry-guided 3D point transformer that refines dense point maps under explicit partial-geometry supervision.
  arXiv  Detail & Related papers  (2026-03-11T18:00:04Z)
• Scalable Adaptation of 3D Geometric Foundation Models via Weak Supervision from Internet Video [76.32954467706581]
  We propose SAGE, a framework for Scalable Adaptation of GEometric foundation models from raw video streams.<n>We use a hierarchical mining pipeline to transform videos into training trajectories and hybrid supervision.<n>Experiments show that SAGE significantly enhances zero-shot generalization, reducing Chamfer Distance by 20-42% on unseen benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T09:53:21Z)
• PLANING: A Loosely Coupled Triangle-Gaussian Framework for Streaming 3D Reconstruction [24.582810014546524]
  We present PLANING, an efficient on-the-fly reconstruction framework built on a hybrid representation.<n>PLANING improves dense mesh Chamfer-L2 by 18.52% over PGSR, surpasses ARTDECO by 1.31 dB PSNR, and reconstructs ScanNetV2 scenes in under 100 seconds.<n>PLANING makes it well suited for a broad range of downstream applications, such as enabling large-scale scene modeling and simulation-ready environments for embodied AI.
  arXiv  Detail & Related papers  (2026-01-29T17:47:26Z)
• GeoVideo: Introducing Geometric Regularization into Video Generation Model [46.38507581500745]
  We introduce geometric regularization losses into video generation by augmenting latent diffusion models with per-frame depth prediction.<n>Our method bridges the gap between appearance generation and 3D structure modeling, leading to improved structural coherence-temporal shape, consistency, and physical plausibility.
  arXiv  Detail & Related papers  (2025-12-03T05:11:57Z)
• St4RTrack: Simultaneous 4D Reconstruction and Tracking in the World [106.91539872943864]
  St4RTrack is a framework that simultaneously reconstructs and tracks dynamic video content in a world coordinate frame from RGB inputs.<n>We predict both pointmaps at the same moment, in the same world, capturing both static and dynamic scene geometry.<n>We establish a new extensive benchmark for world-frame reconstruction and tracking, demonstrating the effectiveness and efficiency of our unified, data-driven framework.
  arXiv  Detail & Related papers  (2025-04-17T17:55:58Z)
• GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priors [47.21120442961684]
  We propose GeometryCrafter, a novel framework that recovers high-fidelity point map sequences with temporal coherence from open-world videos.<n>We show that GeometryCrafter achieves state-of-the-art 3D accuracy, temporal consistency, and generalization capability.
  arXiv  Detail & Related papers  (2025-04-01T17:58:03Z)
• VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment [54.66217340264935]
  VideoLifter is a novel video-to-3D pipeline that leverages a local-to-global strategy on a fragment basis.<n>It significantly accelerates the reconstruction process, reducing training time by over 82% while holding better visual quality than current SOTA methods.
  arXiv  Detail & Related papers  (2025-01-03T18:52:36Z)
• FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models [67.96827539201071]
  We propose a novel test-time optimization approach for 3D scene reconstruction. Our method achieves state-of-the-art cross-dataset reconstruction on five zero-shot testing datasets.
  arXiv  Detail & Related papers  (2023-08-10T17:55:02Z)
• Spatiotemporal Bundle Adjustment for Dynamic 3D Human Reconstruction in the Wild [49.672487902268706]
  We present a framework that jointly estimates camera temporal alignment and 3D point triangulation. We reconstruct 3D motion trajectories of human bodies in events captured by multiple unsynchronized and unsynchronized video cameras.
  arXiv  Detail & Related papers  (2020-07-24T23:50:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.