GHOST: Geometry-Hierarchical Online Streaming Token Eviction for Efficient 3D Reconstruction

• URL: http://arxiv.org/abs/2605.15852v1
• Date: Fri, 15 May 2026 11:07:33 GMT
• Title: GHOST: Geometry-Hierarchical Online Streaming Token Eviction for Efficient 3D Reconstruction
• Abstract summary: Streaming 3D reconstruction from long monocular video sequences requires maintaining a keyvalue (KV) cache.<n>GHOST is a training-free KV cache management framework that exploits the model's own 3D geometry outputs to evict redundant tokens online.<n> Experiments on various benchmarks show that GHOST preserves excellent reconstruction quality while cutting the KV cache by nearly half and delivering 1.75x faster inference.
• Score: 23.110378663548335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Streaming 3D reconstruction from long monocular video sequences requires maintaining a key-value (KV) cache that grows linearly with sequence length, creating a severe memory bottleneck. Existing approaches either truncate the cache to a fixed set of anchor frames, leading to reconstruction quality degradation, or rely on attention-score heuristics that are agnostic to 3D scene structure, failing to preserve geometrically valuable tokens. To address these problems, we present GHOST (Geometry-Hierarchical Online Streaming Token Eviction), a training-free KV cache management framework that exploits the model's own 3D geometry outputs to evict redundant tokens online. GHOST introduces three mutually reinforcing innovations: a hierarchical dual-level importance scoring scheme, a privilege mechanism that protects special tokens from eviction, and a cosine-similarity-guided layer-wise budget allocation. Experiments on various benchmarks show that GHOST preserves excellent reconstruction quality while cutting the KV cache by nearly half and delivering 1.75x faster inference compared to state-of-the-art methods. Our code is available at https://github.com/lokiniuniu/GHOST.

Related papers

• AnyRecon: Arbitrary-View 3D Reconstruction with Video Diffusion Model [29.087865646035695]
  Sparse-view 3D reconstruction is essential for modeling scenes from casual captures, but remain challenging for non-generative reconstruction.<n>We propose AnyRecon, a scalable framework for reconstruction from arbitrary and unordered sparse inputs.<n>Experiments demonstrate robust and scalable reconstruction across irregular inputs, large viewpoint gaps, and long trajectories.
  arXiv  Detail & Related papers  (2026-04-21T17:59:47Z)
• StreamCacheVGGT: Streaming Visual Geometry Transformers with Robust Scoring and Hybrid Cache Compression [39.0869112189715]
  StreamCacheVGGT is a training-free framework that reimagines cache management.<n>It delivers superior reconstruction accuracy and long-term stability while strictly adhering to constant-cost constraints.
  arXiv  Detail & Related papers  (2026-04-16T17:12:10Z)
• STAC: Plug-and-Play Spatio-Temporal Aware Cache Compression for Streaming 3D Reconstruction [16.772404929988507]
  3D reconstruction from streaming inputs requires both long-term temporal consistency and efficient memory usage.<n> STAC is a framework for streaming 3D reconstruction with large causal transformers.<n> experiments show that STAC achieves state-of-the-art reconstruction quality while reducing memory consumption by nearly 10x and accelerating inference by VG 4x.
  arXiv  Detail & Related papers  (2026-03-18T06:36:46Z)
• S2GS: Streaming Semantic Gaussian Splatting for Online Scene Understanding and Reconstruction [57.07346645250984]
  Streaming Semantic Gaussian Splatting (S2GS) is a strictly causal, incremental 3D Gaussian semantic field framework.<n>It does not leverage future frames and continuously updates scene geometry, appearance, and instance-level semantics without reprocessing historical frames.<n>S2GS matches or outperforms strong offline baselines on joint reconstruction-and-understanding benchmarks.
  arXiv  Detail & Related papers  (2026-03-15T05:48:55Z)
• Rewis3d: Reconstruction Improves Weakly-Supervised Semantic Segmentation [58.37682525044409]
  Rewis3d is a framework that leverages recent advances in feed-forward 3D reconstruction to significantly improve weakly supervised semantic segmentation on 2D images.<n>We show that Rewis3d achieves state-of-the-art performance in sparse supervision, outperforming existing approaches by 2-7% without requiring additional labels or inference overhead.
  arXiv  Detail & Related papers  (2026-03-06T15:25:20Z)
• OVGGT: O(1) Constant-Cost Streaming Visual Geometry Transformer [14.628152488797356]
  OVGGT is a training-free framework that bounds both memory and compute to a fixed budget regardless of sequence length.<n>We show that OVGGT processes arbitrarily long videos within a constant VRAM envelope while achieving state-of-the-art 3D geometric accuracy.
  arXiv  Detail & Related papers  (2026-03-06T06:44:17Z)
• PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache [61.57938553036056]
  We introduce PackCache, a training-free KV-cache management method that compacts the KV cache through three coordinated mechanisms.<n>In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences.
  arXiv  Detail & Related papers  (2026-01-07T19:51:06Z)
• OnlineSplatter: Pose-Free Online 3D Reconstruction for Free-Moving Objects [58.38338242973447]
  OnlineSplatter is a novel framework generating high-quality, object-centric 3D Gaussians directly from RGB frames.<n>Our approach anchors reconstruction using the first frame and progressively refines the object representation through a dense Gaussian primitive field.<n>Our core contribution is a dual-key memory module combining latent appearance-geometry keys with explicit directional keys.
  arXiv  Detail & Related papers  (2025-10-23T14:37:25Z)
• GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generation [81.58846231702026]
  We introduce GigaTok, the first approach to improve image reconstruction, generation, and representation learning when scaling visual tokenizers.<n>We identify the growing complexity of latent space as the key factor behind the reconstruction vs. generation dilemma.<n>By scaling to $bf3 space billion$ parameters, GigaTok achieves state-of-the-art performance in reconstruction, downstream AR generation, and downstream AR representation quality.
  arXiv  Detail & Related papers  (2025-04-11T17:59:58Z)
• VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment [54.66217340264935]
  VideoLifter is a novel video-to-3D pipeline that leverages a local-to-global strategy on a fragment basis.<n>It significantly accelerates the reconstruction process, reducing training time by over 82% while holding better visual quality than current SOTA methods.
  arXiv  Detail & Related papers  (2025-01-03T18:52:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.