SAM-MT: Real-Time Interactive Multi-Target Video Segmentation

• URL: http://arxiv.org/abs/2607.08688v1
• Date: Thu, 09 Jul 2026 16:49:57 GMT
• Title: SAM-MT: Real-Time Interactive Multi-Target Video Segmentation
• Abstract summary: Segment Anything 2 (SAM2) is an interactive framework for real-time Multi-Target video segmentation.<n>SAM-MT uses explicit queries to represent different individual targets, in parallel with a shared representation for global context.<n>It successfully decouples latency from the number of targets, achieving real-time speed on par with single-target baselines.
• Score: 46.0193087664104
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern Video Object Segmentation (VOS) involves tracking and segmenting user-specified targets. While recent approaches have achieved remarkable performance in single-target scenarios, extending them to multi-target settings typically involves replicating the single-target processing for each individual object, resulting in reduced frame rates (FPS) with unbounded latency as target count increases. Built upon Segment Anything 2 (SAM2), we propose SAM-MT, which addresses this by transforming the model into an interactive framework for real-time Multi-Target video segmentation. SAM-MT uses explicit queries to represent different individual targets, in parallel with a shared representation for global context. It employs decoupled masked attention to keep individual identities distinct from cross-target interference, and sparse memory for stable temporal evolution, along with specialized strategies for occlusion handling and overlap prevention. SAM-MT successfully decouples latency from the number of targets, achieving real-time speed on par with single-target baselines (>36 FPS for 10 targets) while maintaining SAM2's robust video segmentation performance.

Related papers

• SAM3-DMS: Decoupled Memory Selection for Multi-target Video Segmentation of SAM3 [46.0193087664104]
  Segment Anything 3 (SAM3) has established a powerful foundation that robustly detects, segments, and tracks specified targets in videos.<n>We propose SAM3-DMS, a training-free decoupled strategy that utilizes fine-grained memory selection on individual objects.
  arXiv  Detail & Related papers  (2026-01-14T18:52:14Z)
• Evaluating SAM2 for Video Semantic Segmentation [60.157605818225186]
  The Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos.<n>This paper explores the extension of SAM2 to dense Video Semantic (VSS)<n>Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.
  arXiv  Detail & Related papers  (2025-12-01T15:15:16Z)
• VideoSAM: Open-World Video Segmentation [33.09103541421983]
  We introduce VideoSAM, an end-to-end framework designed to improve object tracking and segmentation consistency in dynamic environments. VideoSAM integrates an agglomerated backbone, RADIO, enabling object association through similarity metrics. Our method is extensively evaluated on the UVO and BURST benchmarks, and robotic videos from RoboTAP, demonstrating its effectiveness and robustness in real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-11T12:56:32Z)
• Training-Free Robust Interactive Video Object Segmentation [82.05906654403684]
  We propose a training-free prompt tracking framework for interactive video object segmentation (I-PT) We jointly adopt sparse points and boxes tracking, filtering out unstable points and capturing object-wise information. Our framework has demonstrated robust zero-shot video segmentation results on popular VOS datasets.
  arXiv  Detail & Related papers  (2024-06-08T14:25:57Z)
• FocSAM: Delving Deeply into Focused Objects in Segmenting Anything [58.042354516491024]
  The Segment Anything Model (SAM) marks a notable milestone in segmentation models. We propose FocSAM with a pipeline redesigned on two pivotal aspects. First, we propose Dynamic Window Multi-head Self-Attention (Dwin-MSA) to dynamically refocus SAM's image embeddings on the target object. Second, we propose Pixel-wise Dynamic ReLU (P-DyReLU) to enable sufficient integration of interactive information from a few initial clicks.
  arXiv  Detail & Related papers  (2024-05-29T02:34:13Z)
• RMP-SAM: Towards Real-Time Multi-Purpose Segment Anything [117.02741621686677]
  This work explores a novel real-time segmentation setting called real-time multi-purpose segmentation.<n>It contains three fundamental sub-tasks: interactive segmentation, panoptic segmentation, and video instance segmentation.<n>We present a novel dynamic convolution-based method, Real-Time Multi-Purpose SAM (RMP-SAM)<n>It contains an efficient encoder and an efficient decoupled adapter to perform prompt-driven decoding.
  arXiv  Detail & Related papers  (2024-01-18T18:59:30Z)
• Segment Anything Meets Point Tracking [116.44931239508578]
  This paper presents a novel method for point-centric interactive video segmentation, empowered by SAM and long-term point tracking. We highlight the merits of point-based tracking through direct evaluation on the zero-shot open-world Unidentified Video Objects (UVO) benchmark. Our experiments on popular video object segmentation and multi-object segmentation tracking benchmarks, including DAVIS, YouTube-VOS, and BDD100K, suggest that a point-based segmentation tracker yields better zero-shot performance and efficient interactions.
  arXiv  Detail & Related papers  (2023-07-03T17:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.