TinySAM 2: Extreme Memory Compression for Efficient Track Anything Model

• URL: http://arxiv.org/abs/2605.18013v1
• Date: Mon, 18 May 2026 08:05:59 GMT
• Title: TinySAM 2: Extreme Memory Compression for Efficient Track Anything Model
• Abstract summary: TinySAM 2 is a lightweight video segmentation model that balances performance and efficiency.<n>Experiments on challenging datasets such as DAVIS and SA-V demonstrate that TinySAM 2 achieves 90% of the performance of SAM 2.1.
• Score: 9.471131843884953
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Segment Anything Model 2 (SAM 2) serves as a core foundation model in the field of video segmentation. Building upon the original SAM model, it introduces a memory bank mechanism and demonstrates outstanding performance in tasks such as semi-supervised video object segmentation and tracking anything. However, the complex computational characteristics of SAM 2's multi-stage image encoder and memory module have raised the barrier to the model's deployment in practical applications. To address this issue, we propose TinySAM 2, a lightweight video segmentation model that balances performance and efficiency. First, a memory quality management mechanism is introduced to select and retain high-informative historical frames as the memory. In addition, a joint-spatial-temporal token compression is proposed that reduces the memory storage and computational cost. Specifically, average pooling is employed to first compress redundancy tokens in the spatial domain. In the temporal domain, informative tokens are selected across frames in the memory bank based on token-level similarity measurement. Besides, we take RepViT as the lightweight image encoder, which further reduces the model parameters. Extensive experiments on challenging datasets such as DAVIS and SA-V demonstrate that TinySAM 2 achieves 90% of the performance of SAM 2.1, with only 7% memory tokens and 3% training data. This study effectively alleviates the bottlenecks in parameter count, computational load, and deployment costs associated with SAM 2, providing a resource-efficient solution for the widespread application of video segmentation models on devices.

Related papers

• Efficient-SAM2: Accelerating SAM2 with Object-Aware Visual Encoding and Memory Retrieval [22.632907736085034]
  Segment Anything Model 2 (SAM2) shows excellent performance in video object segmentation tasks.<n>We propose Efficient-SAM2, which promotes SAM2 to adaptively focus on object regions while eliminating task-irrelevant computations.<n>With negligible additional parameters and minimal training overhead, Efficient-SAM2 delivers 1.68x speedup on SAM2.1-L model with only 1.0% accuracy drop on SA-V test set.
  arXiv  Detail & Related papers  (2026-02-09T02:58:33Z)
• Evaluating SAM2 for Video Semantic Segmentation [60.157605818225186]
  The Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos.<n>This paper explores the extension of SAM2 to dense Video Semantic (VSS)<n>Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.
  arXiv  Detail & Related papers  (2025-12-01T15:15:16Z)
• SAM-I2V: Upgrading SAM to Support Promptable Video Segmentation with Less than 0.2% Training Cost [23.51101492249404]
  We introduce SAM-I2V, an effective image-tovideo upgradation method for cultivating a promptable video segmentation (PVS) model.<n>Our approach strategically upgrades the pre-trained SAM to support PVS, significantly reducing training complexity and resource requirements.
  arXiv  Detail & Related papers  (2025-06-02T04:30:14Z)
• MoSAM: Motion-Guided Segment Anything Model with Spatial-Temporal Memory Selection [21.22536962888316]
  We present MoSAM, incorporating two key strategies to integrate object motion cues into the model and establish more reliable feature memory.<n>MoSAM achieves state-of-the-art results compared to other competitors.
  arXiv  Detail & Related papers  (2025-04-30T02:19:31Z)
• MemorySAM: Memorize Modalities and Semantics with Segment Anything Model 2 for Multi-modal Semantic Segmentation [22.482211353379927]
  Large vision model, AnythingCube Model 2 (SAM2) has shown strong zero-shot segmentation performance on both images and videos.<n>Inspired by cross-frame correlation in videos, we propose to treat multi-modal data as a sequence of frames representing the same scene.<n>Our key idea is to ''memorize'' the modality-agnostic information and'memorize' the semantics related to the targeted scene.
  arXiv  Detail & Related papers  (2025-03-09T17:33:15Z)
• EdgeTAM: On-Device Track Anything Model [65.10032957471824]
  Segment Anything Model (SAM) 2 further extends its capability from image to video inputs through a memory bank mechanism.<n>We aim at making SAM 2 much more efficient so that it even runs on mobile devices while maintaining a comparable performance.<n>We propose EdgeTAM, which leverages a novel 2D Spatial Perceiver to reduce the computational cost.
  arXiv  Detail & Related papers  (2025-01-13T12:11:07Z)
• Look Every Frame All at Once: Video-Ma$^2$mba for Efficient Long-form Video Understanding with Multi-Axis Gradient Checkpointing [52.050036778325094]
  Video-Ma$2$mba is a novel architecture that incorporates State Space Models (SSMs) within the Mamba-2 framework.<n>Our approach significantly reduces the memory footprint compared to standard gradient checkpointing.<n>By maintaining a detailed capture of temporal dynamics, our model improves the accuracy and relevance of responses in long video understanding tasks.
  arXiv  Detail & Related papers  (2024-11-29T04:12:13Z)
• Efficient Track Anything [32.0480507678127]
  Segment Anything Model 2 (SAM 2) has emerged as a powerful tool for video object segmentation and tracking anything.<n>High complexity of multistage image encoder and memory module has limited its applications in real-world tasks.<n>We propose EfficientTAMs, lightweight track anything models that produce high-quality results with low latency and model size.
  arXiv  Detail & Related papers  (2024-11-28T05:52:10Z)
• SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentation [51.90445260276897]
  We prove that the Segment Anything Model 2 (SAM2) can be a strong encoder for U-shaped segmentation models. We propose a simple but effective framework, termed SAM2-UNet, for versatile image segmentation.
  arXiv  Detail & Related papers  (2024-08-16T17:55:38Z)
• FocSAM: Delving Deeply into Focused Objects in Segmenting Anything [58.042354516491024]
  The Segment Anything Model (SAM) marks a notable milestone in segmentation models. We propose FocSAM with a pipeline redesigned on two pivotal aspects. First, we propose Dynamic Window Multi-head Self-Attention (Dwin-MSA) to dynamically refocus SAM's image embeddings on the target object. Second, we propose Pixel-wise Dynamic ReLU (P-DyReLU) to enable sufficient integration of interactive information from a few initial clicks.
  arXiv  Detail & Related papers  (2024-05-29T02:34:13Z)
• TinySAM: Pushing the Envelope for Efficient Segment Anything Model [73.06322749886483]
  We propose a framework to obtain a tiny segment anything model (TinySAM) while maintaining the strong zero-shot performance.<n>With all these proposed methods, our TinySAM leads to orders of magnitude computational reduction and pushes the envelope for efficient segment anything task.
  arXiv  Detail & Related papers  (2023-12-21T12:26:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.