An Efficient Token Compression Framework for Visual Object Tracking

• URL: http://arxiv.org/abs/2605.08329v1
• Date: Fri, 08 May 2026 17:26:24 GMT
• Title: An Efficient Token Compression Framework for Visual Object Tracking
• Abstract summary: Transformer-based trackers leverage a larger number of historical frames to capture richer-temporal cues.<n>This strategy leads to a massive number of visual input tokens.<n>We propose compresstheninteract tracking framework, that learns to efficiently compress template tokens.<n>Our method outperforms current state-of-the-art trackers.
• Score: 37.06665888908352
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Refining visual representations by eliminating their internal feature-level redundancy is crucial for simultaneously optimizing the performance and computational cost of models in visual tracking. To enhance their performance, many contemporary Transformer-based trackers leverage a larger number of historical template frames to capture richer spatio-temporal cues. However, this strategy leads to a massive number of input visual tokens. This creates two critical issues: it imposes a quadratic computational burden and can also degrade the tracker's overall performance. To bridge this gap, we propose a compress-then-interact tracking framework, ETCTrack, that learns to efficiently compress template tokens from historical template frames into a robust target representation, moving beyond handcrafted rules. Our method first employs the Adaptive Token Compressor to dynamically construct compact yet highly discriminative template tokens by filtering out redundant visual tokens. These refined template tokens are then processed by our Hierarchical Interaction Encoder to achieve a deep, adaptive interaction with the search features. Refined search features ensure subsequent precise target localization. Experiments on seven benchmarks demonstrate that our method outperforms current state-of-the-art trackers. ETCTrack-B224 reduces the number of template tokens by 60%, leading to a 21.4% reduction in MACs with only a 0.4% drop in accuracy. The source code are available at https://github.com/PJD-WJ/ETCTrack.

Related papers

• UTPTrack: Towards Simple and Unified Token Pruning for Visual Tracking [23.83535022949326]
  One-stream Transformer-based trackers achieve advanced performance in visual object tracking but suffer from significant computational overhead.<n>We introduce UTPTrack, a simple and Unified Token Pruning framework that, for the first time, jointly compresses all three components.<n>Extensive evaluations on 10 benchmarks demonstrate that UTPTrack achieves a new state-of-the-art in the accuracy-efficiency trade-off for pruning-based trackers.
  arXiv  Detail & Related papers  (2026-02-27T06:58:09Z)
• TrajTok: Learning Trajectory Tokens enables better Video Understanding [63.1260672430712]
  Tokenization in video models, typically through patchification, generates an excessive and redundant number of tokens.<n>We propose TrajTok, an end-to-end video tokenizer module that is fully integrated and co-trained with video models for a downstream objective.<n>We show that it can be seamlessly integrated as either a probing head for pretrained visual features (TrajAdapter) or an alignment connector in vision-language models (TrajVLM) with especially strong performance in long-video reasoning.
  arXiv  Detail & Related papers  (2026-02-26T09:15:34Z)
• CORE: Compact Object-centric REpresentations as a New Paradigm for Token Merging in LVLMs [29.08277140543501]
  We introduce CORE (Compact Object-centric REpresentations), a new paradigm for visual token compression.<n> CORE leverages an efficient segmentation decoder to generate object masks, which serve as a high-level semantic prior to guide the merging of visual tokens.<n>Experiments show that CORE not only establishes a new state-of-the-art on six authoritative benchmarks for fixed-rate compression, but also achieves dramatic efficiency gains in adaptive-rate settings.
  arXiv  Detail & Related papers  (2025-11-18T03:02:23Z)
• Efficient Token Compression for Vision Transformer with Spatial Information Preserved [59.79302182800274]
  Token compression is essential for reducing the computational and memory requirements of transformer models.<n>We propose an efficient and hardware-compatible token compression method called Prune and Merge.
  arXiv  Detail & Related papers  (2025-03-30T14:23:18Z)
• Two-stream Beats One-stream: Asymmetric Siamese Network for Efficient Visual Tracking [54.124445709376154]
  We propose a novel asymmetric Siamese tracker named textbfAsymTrack for efficient tracking.<n>Building on this architecture, we devise an efficient template modulation mechanism to inject crucial cues into the search features.<n>Experiments demonstrate that AsymTrack offers superior speed-precision trade-offs across different platforms.
  arXiv  Detail & Related papers  (2025-03-01T14:44:54Z)
• Explicit Visual Prompts for Visual Object Tracking [23.561539973210248]
  textbfEVPTrack is a visual tracking framework that exploits explicit visual prompts between consecutive frames. We show that our framework can achieve competitive performance at a real-time by exploiting both explicit and multi-scale information.
  arXiv  Detail & Related papers  (2024-01-06T07:12:07Z)
• AiluRus: A Scalable ViT Framework for Dense Prediction [95.1313839257891]
  Vision transformers (ViTs) have emerged as a prevalent architecture for vision tasks owing to their impressive performance. We propose to apply adaptive resolution for different regions in the image according to their importance. We evaluate our proposed method on three different datasets and observe promising performance.
  arXiv  Detail & Related papers  (2023-11-02T12:48:43Z)
• Revisiting Token Pruning for Object Detection and Instance Segmentation [25.3324628669201]
  We investigate token pruning to accelerate inference for object and instance segmentation. We show a reduction in performance decline from 1.5 mAP to 0.3 mAP in both boxes and masks, compared to existing token pruning methods.
  arXiv  Detail & Related papers  (2023-06-12T11:55:33Z)
• Compact Transformer Tracker with Correlative Masked Modeling [16.234426179567837]
  Transformer framework has been showing superior performances in visual object tracking. Recent advances focus on exploring attention mechanism variants for better information aggregation. In this paper, we prove that the vanilla self-attention structure is sufficient for information aggregation.
  arXiv  Detail & Related papers  (2023-01-26T04:58:08Z)
• Learning Dynamic Compact Memory Embedding for Deformable Visual Object Tracking [82.34356879078955]
  We propose a compact memory embedding to enhance the discrimination of the segmentation-based deformable visual tracking method. Our method outperforms the excellent segmentation-based trackers, i.e., D3S and SiamMask on DAVIS 2017 benchmark.
  arXiv  Detail & Related papers  (2021-11-23T03:07:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.