Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction

• URL: http://arxiv.org/abs/2605.06270v1
• Date: Thu, 07 May 2026 13:45:37 GMT
• Title: Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction
• Abstract summary: Feed-forward 3D reconstruction models based on Vision Transformers can estimate scene geometry and camera poses from a small set of input images.<n>Scaling them to video inputs with hundreds or thousands of frames remains challenging due to the quadratic cost of global attention layers.<n>We propose Spark3R, a training-free acceleration framework that decouples the compression of query tokens and key-value tokens.<n>As a plug-and-play framework requiring no retraining, Spark3R integrates directly into multiple pretrained feed-forward 3D reconstruction models.
• Score: 47.55355615080919
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Feed-forward 3D reconstruction models based on Vision Transformers can directly estimate scene geometry and camera poses from a small set of input images, but scaling them to video inputs with hundreds or thousands of frames remains challenging due to the quadratic cost of global attention layers. Recent token-merging methods accelerate these models by compressing the token sequence within the global attention layers, but they apply a uniform reduction to query tokens and key-value tokens, ignoring their functionally distinct roles in 3D reconstruction. In this work, we identify a key property of feed-forward 3D reconstruction models: query tokens encode view-specific geometric requests and are sensitive to compression, while key-value tokens represent shared scene context and tolerate aggressive compression. Guided by this insight, we propose Spark3R, a training-free acceleration framework that decouples the compression of query tokens and key-value tokens by assigning distinct reduction factors, with intra-group token merging applied to query tokens and lightweight token pruning to key-value tokens. Additionally, Spark3R adaptively adjusts the key-value reduction factor across layers, further improving the quality-efficiency trade-off. As a plug-and-play framework requiring no retraining, Spark3R integrates directly into multiple pretrained feed-forward 3D reconstruction models, including VGGT, $π^3$, and Depth-Anything-3, and achieves up to $28\times$ speedup on 1,000-frame inputs while maintaining competitive reconstruction quality.

Related papers

• ZipTok3D: High-Fidelity 3D Tokenization with Compact Token Prefixes [38.735159381505014]
  ZipTok3D is a 3D tokenizer designed for high-fidelity reconstruction from extremely short token sequences.<n>With the same token dimension, ZipTok3D achieves reconstruction quality comparable to the 32-token COD-VAE baseline.
  arXiv  Detail & Related papers  (2026-09-01T18:07:49Z)
• See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model [25.03258383353867]
  Differentiable Grid Sampler (GridS) is a plug-and-play module that performs task-aware, continuous resampling of visual tokens in Vision-Language-Action (VLA) models.<n>GridS preserves essential spatial information while achieving drastic compression with fewer than 10% original visual tokens.
  arXiv  Detail & Related papers  (2026-05-12T09:08:42Z)
• Speed3R: Sparse Feed-forward 3D Reconstruction Models [15.534969350839788]
  We introduce Speed3R, an end-to-end trainable model inspired by the core principle of Structure-from-Motion.<n>Speed3R features a dual-branch attention mechanism where a compression branch creates a coarse contextual prior to guide a selection branch.<n>This strategy mimics the efficiency of traditional keypoint matching, achieving a remarkable 12.4x inference speedup on 1000-view sequences.
  arXiv  Detail & Related papers  (2026-03-09T07:46:51Z)
• SceneTok: A Compressed, Diffusable Token Space for 3D Scenes [18.094424146923892]
  We present SceneTok, a novel tokenizer for encoding view sets of scenes into a compressed and diffusable set of unstructured tokens.<n>We show that the compression is 1-3 orders of magnitude stronger than for other representations while still reaching state-of-the-art reconstruction quality.
  arXiv  Detail & Related papers  (2026-02-21T16:08:17Z)
• PIO-FVLM: Rethinking Training-Free Visual Token Reduction for VLM Acceleration from an Inference-Objective Perspective [59.24570811503256]
  We propose PIO-FVLM to reduce redundant visual tokens in vision-models (VLMs) to accelerate inference.<n>The proposed PIO-FVLM is training-free and compatible with FlashAttention, friendly to practical application and deployment.<n>On LLaVA-Next-7B, PIO-FVLM retains just 11.1% of visual tokens but maintains 97.2% of the original performance.
  arXiv  Detail & Related papers  (2026-02-04T15:33:10Z)
• Behavior-Equivalent Token: Single-Token Replacement for Long Prompts in LLMs [55.827877498548965]
  We propose a lightweight training framework that learns a single prompt-specific Behavior-Equivalent token ([BE])<n>The framework first trains [BE] to encode the natural-language content of the original system prompt via reconstruction, and then distills the prompt's downstream behavior into this single token.<n> Empirical evaluations on three datasets show that a single [BE] token achieves up to a 3000x reduction in prompt length, while retaining about 98% of the downstream performance of the original system prompts.
  arXiv  Detail & Related papers  (2025-11-28T15:22:52Z)
• H$_{2}$OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformers [124.11648300910444]
  We present a hierarchical plug-and-play pruning-and-$-recovering framework, called Hierarchical Hourglass Tokenizer (H$_2$OT)<n>Our method is general-purpose: it can be easily incorporated into common VPT models on both seq2seq and seq2frame pipelines.
  arXiv  Detail & Related papers  (2025-09-08T17:59:59Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)
• CLiFT: Compressive Light-Field Tokens for Compute-Efficient and Adaptive Neural Rendering [20.916387646203273]
  This paper proposes a neural rendering approach that represents a scene as "compressed light-field tokens (CLiFTs)"<n>CLiFT enables compute-efficient rendering by compressed tokens, while being capable of changing the number of tokens to represent a scene or render a novel view with one trained network.
  arXiv  Detail & Related papers  (2025-07-11T17:38:52Z)
• GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generation [81.58846231702026]
  We introduce GigaTok, the first approach to improve image reconstruction, generation, and representation learning when scaling visual tokenizers.<n>We identify the growing complexity of latent space as the key factor behind the reconstruction vs. generation dilemma.<n>By scaling to $bf3 space billion$ parameters, GigaTok achieves state-of-the-art performance in reconstruction, downstream AR generation, and downstream AR representation quality.
  arXiv  Detail & Related papers  (2025-04-11T17:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.