HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction

• URL: http://arxiv.org/abs/2605.23889v1
• Date: Fri, 22 May 2026 17:50:48 GMT
• Title: HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction
• Abstract summary: Existing methods often suffer from drift, jitter, or collapse on long sequences.<n>We propose HorizonStream, a long-horizon Transformer that explicitly factorizes this kernel.<n>Experiments show that HorizonStream generalizes stably to sequences exceeding 10,000 frames with constant memory and linear time.
• Score: 18.749241400724493
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online 3D reconstruction requires estimating camera pose and scene geometry under strict causal and bounded-memory constraints. Existing methods often suffer from drift, jitter, or collapse on long sequences. We trace these failures to a fundamental mismatch. Streaming geometry is inherently temporally heterogeneous, with evidence ranging from short-lived correspondences to persistent global scale. However, current architectures impose uniform and pathological influence patterns. For example, sliding windows enforce hard cutoffs, while ungated recurrence and causal attention cause cache saturation and spike-like attention sinks. To resolve this, we formalize geometric propagation as an \emph{evidence influence kernel} and propose HorizonStream, a long-horizon Transformer that explicitly factorizes this kernel. For the long-range temporal factor, Geometric Linear Attention learns channel-wise decay rates to enable bounded, multi-timescale propagation of geometric evidence. For the short-range spatial factor, Geometric Local Attention with Spatiotemporal RoPE performs reliable 3D matching while suppressing attention sinks. Finally, Metric Readout Tokens recover stable scale and rigid pose directly from the persistent geometric state. Extensive experiments show that HorizonStream, trained on only 48-frame clips, generalizes stably to sequences exceeding 10,000\ frames with constant memory and linear time, achieving state-of-the-art streaming 3D reconstruction performance. Project Page: https://3dagentworld.github.io/horizonstream/

Related papers

• Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction [10.383154504465232]
  ABot-Recon is a simple streaming model that caches KV features from only the preceding 11 frames.<n>It predicts a point map in the current camera coordinate system together with an adjacent-frame relative pose.<n>To reduce accumulated drift, a lightweight temporal refiner improves relative rotations using recent visual and motion context.
  arXiv  Detail & Related papers  (2026-08-27T15:07:38Z)
• VideoWeave: Unlocking Geometric Consistency in Video Generation via Joint Geometry-Video Modeling [49.98112719317813]
  Video diffusion models often fail to preserve 3D structure over time.<n>Existing methods usually enforce geometric consistency by using explicit geometry reconstructions.<n>We propose VideoWeave, a latent-space post-training framework that uses implicit geometry-model features to constrain the generative distribution.
  arXiv  Detail & Related papers  (2026-06-12T06:41:13Z)
• S2GS: Streaming Semantic Gaussian Splatting for Online Scene Understanding and Reconstruction [57.07346645250984]
  Streaming Semantic Gaussian Splatting (S2GS) is a strictly causal, incremental 3D Gaussian semantic field framework.<n>It does not leverage future frames and continuously updates scene geometry, appearance, and instance-level semantics without reprocessing historical frames.<n>S2GS matches or outperforms strong offline baselines on joint reconstruction-and-understanding benchmarks.
  arXiv  Detail & Related papers  (2026-03-15T05:48:55Z)
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories [78.78355829813793]
  Existing memory-based approaches often condition generation on globally reconstructed 3D scenes by rendering anchor videos from the reconstructed geometry in the history.<n>We introduce AnchorWeave, a memory-augmented video generation framework that replaces a single misaligned global memory with multiple clean local geometric memories.<n>Experiments demonstrate that AnchorWeave significantly improves long-term scene consistency while maintaining strong visual quality.
  arXiv  Detail & Related papers  (2026-02-16T17:23:08Z)
• LongStream: Long-Sequence Streaming Autoregressive Visual Geometry [18.46781332515933]
  LongStream is a novel gauge-decoupled streaming visual geometry model for metric-scale scene reconstruction.<n>It reformulates long-range extrapolation into a constant-difficulty local task.<n>It delivers stable, metric-scale reconstruction over kilometer-scale sequences at 18 FPS.
  arXiv  Detail & Related papers  (2026-02-13T18:30:51Z)
• Geometry-Aware Rotary Position Embedding for Consistent Video World Model [48.914346802616414]
  ViewRope is a geometry-aware encoding that injects camera-ray directions directly into video transformer self-attention layers.<n>Geometry-Aware Frame-Sparse Attention exploits these geometric cues to selectively attend to relevant historical frames.<n>Our results demonstrate that ViewRope substantially improves long-term consistency while reducing computational costs.
  arXiv  Detail & Related papers  (2026-02-08T08:01:16Z)
• LoL: Longer than Longer, Scaling Video Generation to Hour [50.945885467651216]
  This work achieves the first demonstration of real-time, streaming, and infinite-length video generation with little quality decay.<n>As an illustration, we generate continuous videos up to 12 hours in length, which, to our knowledge, is among the longest publicly demonstrated results in streaming video generation.
  arXiv  Detail & Related papers  (2026-01-23T17:21:35Z)
• InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams [22.277387867568834]
  InfiniteVGGT is a causal visual geometry transformer that operationalizes the concept of a rolling memory through a bounded yet adaptive and perpetually expressive KV cache.<n>We introduce the Long3D benchmark, which, for the first time, enables a rigorous evaluation of continuous 3D geometry estimation on sequences about 10,000 frames.
  arXiv  Detail & Related papers  (2026-01-05T17:11:00Z)
• WorldWarp: Propagating 3D Geometry with Asynchronous Video Diffusion [78.20778143251171]
  WorldWarp is a framework that couples a 3D structural anchor with a 2D generative refiner.<n>WorldWarp maintains consistency across video chunks by dynamically updating the 3D cache at every step.<n>It achieves state-of-the-art fidelity by ensuring that 3D logic guides structure while diffusion logic perfects texture.
  arXiv  Detail & Related papers  (2025-12-22T18:53:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.