OVGGT: O(1) Constant-Cost Streaming Visual Geometry Transformer

• URL: http://arxiv.org/abs/2603.05959v2
• Date: Mon, 09 Mar 2026 12:16:48 GMT
• Title: OVGGT: O(1) Constant-Cost Streaming Visual Geometry Transformer
• Abstract summary: OVGGT is a training-free framework that bounds both memory and compute to a fixed budget regardless of sequence length.<n>We show that OVGGT processes arbitrarily long videos within a constant VRAM envelope while achieving state-of-the-art 3D geometric accuracy.
• Score: 14.628152488797356
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing 3D geometry from streaming video requires continuous inference under bounded resources. Recent geometric foundation models achieve impressive reconstruction quality through all-to-all attention, yet their quadratic cost confines them to short, offline sequences. Causal-attention variants such as StreamVGGT enable single-pass streaming but accumulate an ever-growing KV cache, exhausting GPU memory within hundreds of frames and precluding the long-horizon deployment that motivates streaming inference in the first place. We present OVGGT, a training-free framework that bounds both memory and compute to a fixed budget regardless of sequence length. Our approach combines Self-Selective Caching, which leverages FFN residual magnitudes to compress the KV cache while remaining fully compatible with FlashAttention, with Dynamic Anchor Protection, which shields coordinate-critical tokens from eviction to suppress geometric drift over extended trajectories. Extensive experiments on indoor, outdoor, and ultra-long sequence benchmarks demonstrate that OVGGT processes arbitrarily long videos within a constant VRAM envelope while achieving state-of-the-art 3D geometric accuracy.

Related papers

• Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding [80.10744910189997]
  multimodal large language models typically serialize trajectories autoregressively.<n>We introduce Parallel Tube Decoding (PTD), a generative formulation that decomposes grounding into a temporal block.<n>On VidSTG, PTD Tube reduces Completion by 79x and increases spatial decoding by 92x over standard autoregressive decoding.
  arXiv  Detail & Related papers  (2026-08-28T11:05:30Z)
• Mamba-VGGT: Persistent Long-Sequence Video Geometry Grounded Transformer via External Sliding Window Mamba Memory [37.05996890779604]
  We present Mamba-VGGT, an enhanced VGGT framework capable of persistent long-range reasoning.<n>Our work provides a scalable, linear-complexity solution for geometry-grounded world modeling in extensive 3D environments.
  arXiv  Detail & Related papers  (2026-05-17T14:34:13Z)
• Attention Itself Could Retrieve.RetrieveVGGT: Training-Free Long Context Streaming 3D Reconstruction via Query-Key Similarity Retrieval [86.05617542701644]
  Visual Geometry Grounded Transformer (VGGT) advances 3D reconstruction via scalable Transformer architecture.<n>StreamVGGT enables streaming with causal attention, yet its KV cache grows linearly with frames, causing memory overflow and quality degradation.<n>We present RetrieveVGGT, a training-free framework, which formulates context construction for VGGT as a retrieval problem.
  arXiv  Detail & Related papers  (2026-05-10T16:41:54Z)
• FrameVGGT: Frame Evidence Rolling Memory for streaming VGGT [3.5844726281072448]
  Streaming Visual Geometry Transformers such as StreamVGGT enable strong online 3D perception but suffer from unbounded KV-cache growth.<n>We propose FrameVGGT, a frame-driven rolling explicit-memory framework that treats each frame's incremental KV contribution as a coherent evidence block.
  arXiv  Detail & Related papers  (2026-03-08T15:46:03Z)
• LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory [97.14005794889134]
  We present LoGeR, a novel architecture that scales dense 3D reconstruction to extremely long sequences without post-optimization.<n>LoGeR processes video streams in chunks, leveraging strong bidirectional priors for high-fidelity intra-chunk reasoning.<n>This memory architecture enables LoGeR to be trained on sequences of 128 frames, and generalize up to thousands of frames during inference.
  arXiv  Detail & Related papers  (2026-03-03T18:55:37Z)
• XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression [20.18561757219652]
  XStreamVGGT is a tuning-free approach that seamlessly integrates pruning and quantization to systematically compress the KV cache.<n>XStreamVGGT achieves mostly negligible performance degradation while substantially reducing memory usage by 4.42$times$.
  arXiv  Detail & Related papers  (2026-02-25T11:02:02Z)
• Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization [83.406036390582]
  Quant VideoGen (QVG) is a training free KV cache quantization framework for autoregressive video diffusion models.<n>It reduces KV memory by up to 7.0 times with less than 4% end to end latency overhead.<n>It consistently outperforms existing baselines in generation quality.
  arXiv  Detail & Related papers  (2026-02-03T00:54:32Z)
• Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformers [95.68243351895107]
  We propose a holistic, video-centric paradigm named textbfLocal textbfDiffusion textbfForcing for textbfVideo textbfFrame textbfInterpolation (LDF-VFI)<n>Our framework is built upon an auto-regressive diffusion transformer that models the entire video sequence to ensure long-range temporal coherence.<n>LDF-VFI achieves state-of-the-art performance on challenging long-sequence benchmarks, demonstrating superior per
  arXiv  Detail & Related papers  (2026-01-21T12:58:52Z)
• PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache [61.57938553036056]
  We introduce PackCache, a training-free KV-cache management method that compacts the KV cache through three coordinated mechanisms.<n>In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences.
  arXiv  Detail & Related papers  (2026-01-07T19:51:06Z)
• InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams [22.277387867568834]
  InfiniteVGGT is a causal visual geometry transformer that operationalizes the concept of a rolling memory through a bounded yet adaptive and perpetually expressive KV cache.<n>We introduce the Long3D benchmark, which, for the first time, enables a rigorous evaluation of continuous 3D geometry estimation on sequences about 10,000 frames.
  arXiv  Detail & Related papers  (2026-01-05T17:11:00Z)
• XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression [20.18561757219652]
  XStreamVGGT is a tuning-free approach that compresses the KV cache through joint pruning and quantization.<n>We show that XStreamVGGT achieves mostly negligible performance degradation while substantially reducing memory usage.
  arXiv  Detail & Related papers  (2026-01-03T14:59:50Z)
• RELIC: Interactive Video World Model with Long-Horizon Memory [74.81433479334821]
  A truly interactive world model requires real-time long-horizon streaming, consistent spatial memory, and precise user control.<n>We present RELIC, a unified framework that tackles these three challenges altogether.<n>Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time.
  arXiv  Detail & Related papers  (2025-12-03T18:29:20Z)
• AutoScape: Geometry-Consistent Long-Horizon Scene Generation [69.2451355181344]
  AutoScape is a long-horizon driving scene generation framework.<n>It generates realistic and geometrically consistent driving videos of over 20 seconds.<n>It improves the long-horizon FID and FVD scores over the prior state-of-the-art by 48.6% and 43.0%, respectively.
  arXiv  Detail & Related papers  (2025-10-23T16:44:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.