Evict3R: Training-Free Token Eviction for Memory-Bounded Streaming Visual Geometry Transformers

• URL: http://arxiv.org/abs/2509.17650v2
• Date: Thu, 02 Oct 2025 18:38:00 GMT
• Title: Evict3R: Training-Free Token Eviction for Memory-Bounded Streaming Visual Geometry Transformers
• Abstract summary: Streaming visual transformers like StreamVGGT achieve strong 3D perception but suffer from growth of key value (KV) memory.<n>We propose a training-free, inference-time token eviction policy that bounds memory by discarding redundant tokens while keeping the most informative ones.
• Score: 1.9268905951820923
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Streaming visual transformers like StreamVGGT achieve strong 3D perception but suffer from unbounded growth of key value (KV) memory, which limits scalability. We propose a training-free, inference-time token eviction policy that bounds memory by discarding redundant tokens while keeping the most informative ones. Our method uses significantly less memory with little to no drop in accuracy: on 7-Scenes with long sequences it reduces peak memory from 18.63 GB to 9.39 GB while accuracy and completeness drop by only 0.003. Under strict memory budgets, eviction enables denser frame sampling, which improves reconstruction accuracy compared to the baseline. Experiments across video depth estimation (Sintel, KITTI), 3D reconstruction (7-Scenes, NRGBD), and camera pose estimation (Sintel, TUM-dynamics) show that our approach closely matches StreamVGGT at a fraction of the memory and makes long-horizon streaming inference more practical.

Related papers

• EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation [81.54723508469617]
  EvoMemNav is an efficient, self-evolving, fine-grained memory framework for zero-shot embodied navigation.<n>VSMGraph organizes views with semantic cues and topological relations into a room-view-object hierarchy.<n>Experiments on GOAT-Bench and HM3D across object, text-description, and image-goal modalities show consistent gains in SR/SPL.
  arXiv  Detail & Related papers  (2026-06-02T11:27:44Z)
• Attention Itself Could Retrieve.RetrieveVGGT: Training-Free Long Context Streaming 3D Reconstruction via Query-Key Similarity Retrieval [86.05617542701644]
  Visual Geometry Grounded Transformer (VGGT) advances 3D reconstruction via scalable Transformer architecture.<n>StreamVGGT enables streaming with causal attention, yet its KV cache grows linearly with frames, causing memory overflow and quality degradation.<n>We present RetrieveVGGT, a training-free framework, which formulates context construction for VGGT as a retrieval problem.
  arXiv  Detail & Related papers  (2026-05-10T16:41:54Z)
• Semantic-Aware Adaptive Visual Memory for Streaming Video Understanding [55.7992006853979]
  SAVEMem is a framework that brings semantic awareness into memory generation and lets the retrieval scope adapt per query.<n>We present SAVEMem, a training-free dual-stage framework that brings semantic awareness into memory generation and lets the retrieval scope adapt per query.
  arXiv  Detail & Related papers  (2026-05-08T15:40:40Z)
• Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction [50.5449251266956]
  This paper addresses the task of large-scale 3D scene reconstruction from long video sequences.<n>Recent feed-forward reconstruction models have shown promising results by directly regressing 3D geometry from RGB images without explicit 3D priors or geometric constraints.<n>We propose a novel neural global context representation that efficiently compresses and retains long-range scene information.
  arXiv  Detail & Related papers  (2026-04-09T17:59:50Z)
• Mem3R: Streaming 3D Reconstruction with Hybrid Memory via Test-Time Training [8.037339305506656]
  We propose Mem3R, a streaming 3D reconstruction model with a hybrid memory design.<n>For camera tracking, Mem3R employs an implicit fast-weight memory implemented as a lightweight Multi-Layer Perceptron.<n>For geometric mapping, Mem3R maintains an explicit token-based fixed-size state.
  arXiv  Detail & Related papers  (2026-04-08T16:41:22Z)
• From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents [78.30630000529133]
  We propose MM-Mem, a pyramidal multimodal memory architecture grounded in Fuzzy-Trace Theory.<n> MM-Mem memory structures hierarchically into a Sensory Buffer, Episodic Stream, and Symbolic.<n>Experiments confirm the effectiveness of MM-Mem on both offline and streaming tasks.
  arXiv  Detail & Related papers  (2026-03-02T05:12:45Z)
• XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression [20.18561757219652]
  XStreamVGGT is a tuning-free approach that seamlessly integrates pruning and quantization to systematically compress the KV cache.<n>XStreamVGGT achieves mostly negligible performance degradation while substantially reducing memory usage by 4.42$times$.
  arXiv  Detail & Related papers  (2026-02-25T11:02:02Z)
• Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memory [50.30283773196725]
  Existing approaches rely on key-value caching to accumulate frame-level details over time, but use a limited number of tokens per frame.<n>We propose scaling the token budget to enable more granular-temporal understanding and reasoning.
  arXiv  Detail & Related papers  (2026-02-20T18:59:50Z)
• XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression [20.18561757219652]
  XStreamVGGT is a tuning-free approach that compresses the KV cache through joint pruning and quantization.<n>We show that XStreamVGGT achieves mostly negligible performance degradation while substantially reducing memory usage.
  arXiv  Detail & Related papers  (2026-01-03T14:59:50Z)
• SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning [33.149133156465474]
  Long-context inference in large language models is increasingly constrained by the KV cache bottleneck.<n>We propose SPARK, a training-free plug-and-play method that applies unstructured sparsity by pruning KV at the channel level.<n>By reducing channel-level redundancy, SPARK enables processing of longer sequences within the same memory budget.
  arXiv  Detail & Related papers  (2025-08-21T03:48:28Z)
• METok: Multi-Stage Event-based Token Compression for Efficient Long Video Understanding [41.60539587719931]
  We propose METok, a training-free, Multi-stage Event-based Token compression framework.<n>We show METok achieves an optimal trade-off between efficiency and accuracy by dynamically selecting informative visual tokens.<n>For instance, equipping LongVA-7B with METok realizes an 80.6% FLOPs reduction and 93.5% KV Cache memory savings.
  arXiv  Detail & Related papers  (2025-06-03T13:19:41Z)
• Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMs [6.222287867011644]
  We propose MorphKV, an inference-time technique that maintains a constant-sized KV cache while preserving accuracy.<n>Unlike retention or lossy compression, MorphKV iteratively refines the KV cache via lightweight updates guided by attention patterns of recent tokens.<n>Our studies show 52.9$%$ memory savings and 18.2$%$ higher accuracy on average compared to state-of-the-art prior works.
  arXiv  Detail & Related papers  (2025-03-02T18:12:50Z)
• Efficient Video Object Segmentation via Modulated Cross-Attention Memory [123.12273176475863]
  We propose a transformer-based approach, named MAVOS, to model temporal smoothness without requiring frequent memory expansion. Our MAVOS achieves a J&F score of 63.3% while operating at 37 frames per second (FPS) on a single V100 GPU.
  arXiv  Detail & Related papers  (2024-03-26T17:59:58Z)
• Recurrent Dynamic Embedding for Video Object Segmentation [54.52527157232795]
  We propose a Recurrent Dynamic Embedding (RDE) to build a memory bank of constant size. We propose an unbiased guidance loss during the training stage, which makes SAM more robust in long videos. We also design a novel self-correction strategy so that the network can repair the embeddings of masks with different qualities in the memory bank.
  arXiv  Detail & Related papers  (2022-05-08T02:24:43Z)
• Mesa: A Memory-saving Training Framework for Transformers [58.78933015299703]
  We present Mesa, a memory-saving training framework for Transformers. Mesa uses exact activations during forward pass while storing a low-precision version of activations to reduce memory consumption during training. Experiments on ImageNet, CIFAR-100 and ADE20K demonstrate that Mesa can reduce half of the memory footprints during training.
  arXiv  Detail & Related papers  (2021-11-22T11:23:01Z)
• Memformer: A Memory-Augmented Transformer for Sequence Modeling [55.780849185884996]
  We present Memformer, an efficient neural network for sequence modeling. Our model achieves linear time complexity and constant memory space complexity when processing long sequences.
  arXiv  Detail & Related papers  (2020-10-14T09:03:36Z)
• TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning [78.80707950262214]
  On-device learning enables edge devices to continually adapt the AI models to new data. Existing work solves this problem by reducing the number of trainable parameters. We present Tiny-Transfer-Learning (TinyTL) for memory-efficient on-device learning.
  arXiv  Detail & Related papers  (2020-07-22T18:39:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.