EOVSAM: Efficient Open-Vocabulary Segmentation with SAM 3 in One Pass

• URL: http://arxiv.org/abs/2608.02284v1
• Date: Mon, 03 Aug 2026 14:15:37 GMT
• Title: EOVSAM: Efficient Open-Vocabulary Segmentation with SAM 3 in One Pass
• Abstract summary: We propose an Efficient Open-Vocabulary segmentation framework with SAM 3 (EOVSAM)<n>EOVSAM adapts SAM 3 for single-pass prediction.<n>It consistently improves segmentation accuracy over vanilla SAM 3 on all evaluated datasets.
• Score: 79.69317943751223
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Open-vocabulary segmentation identifies and segments objects from arbitrary textual descriptions. SAM 3 supports noun-phrase-guided segmentation and achieves competitive open-vocabulary performance through exhaustive vocabulary traversal, yet suffers from prohibitive computational overhead as target categories scale. In this paper, we propose an Efficient Open-Vocabulary segmentation framework with SAM 3 (EOVSAM), which adapts SAM 3 for single-pass prediction. EOVSAM removes prompt conditioning to turn SAM 3 into an efficient mask generator and introduces a new Attentional Aggregation strategy to optimize open-vocabulary classification end-to-end. This formulation avoids the multi-stage pipelines and post-processing heuristics commonly used by existing methods, while mitigating the closed-set collapse that can arise when classification is optimized directly. EOVSAM consistently improves segmentation accuracy over vanilla SAM 3 on all evaluated datasets and accelerates inference by up to 338$\times$. Furthermore, EOVSAM maintains high accuracy at lower resolutions while achieving even more remarkable inference speeds. Experiments on standard semantic and panoptic segmentation benchmarks show that EOVSAM combines competitive or state-of-the-art accuracy with a substantial speed advantage over existing open-vocabulary segmentation models. Code and models are available at https://github.com/hustvl/EOVSAM.

Related papers

• ActiveSAM: Image-Conditional Class Pruning for Fast and Accurate Open-Vocabulary Segmentation [40.432049062995326]
  We introduce ActiveSAM, a training-free, zero-shot inference framework that turns SAM 3 into an active-vocabulary segmenter.<n>Across eight OVSS benchmarks, ActiveSAM improves the speed-accuracy tradeoff of training-free open-vocabulary semantic segmentation.<n>ActiveSAM is well-suited for deployment in noisy-input domains such as autonomous driving and embodied AI.
  arXiv  Detail & Related papers  (2026-06-15T17:31:30Z)
• CLIP-Guided SAM: Parameter-Efficient Semantic Conditioning for Promptable Segmentation [6.517222960194991]
  We propose CLIP-Guided SAM, a parameter-efficient segmentation framework built on internal semantic conditioning.<n>Instead of using semantic signals only to generate prompts, we inject CLIP-derived text, vision, and similarity features directly into SAM's image encoder.<n>Our framework is designed for low labeled-data settings and applies to both general-domain benchmarks and specialized downstream tasks.
  arXiv  Detail & Related papers  (2026-05-24T01:40:30Z)
• Semantic-Fast-SAM: Efficient Semantic Segmenter [0.6816905600359814]
  We propose a semantic segmentation framework that combines the Fast Segment Anything model with a semantic labeling pipeline to achieve real-time performance without sacrificing accuracy.<n>FastSAM is an efficient CNN-based re-implementation of the Segment Anything Model (SAM) that runs much faster than the original transformer-based SAM.<n>SFS produces high-quality semantic segmentation maps at a fraction of the computational cost and memory footprint of the original SAM-based approach.
  arXiv  Detail & Related papers  (2026-04-22T04:18:39Z)
• Do Instance Priors Help Weakly Supervised Semantic Segmentation? [82.75875820720616]
  SeSAM is a framework that uses a foundational segmentation model, i.e. Segment Anything Model (SAM), with weak labels.<n>SeSAM decomposes class masks into connected components, samples point prompts along object skeletons, selects SAM masks using weak-label coverage, and iteratively refines labels using pseudo-labels.<n> integrated with a semi-supervised learning framework, SeSAM balances ground-truth labels, SAM-based pseudo-labels, and high-confidence pseudo-labels.
  arXiv  Detail & Related papers  (2026-04-13T08:29:49Z)
• Tarot-SAM3: Training-free SAM3 for Any Referring Expression Segmentation [18.568343383992072]
  We present Tarot-SAM3, a training-free framework that can accurately segment from any referring expression.<n>Tarot-SAM3 consists of two key phases. First, the Reasoning Expression Interpreter (ERI) phase introduces reasoning-assisted prompt options.<n>Second, the Mask Self-Refining (MSR) phase selects the best mask across prompt types and performs self-refinement.
  arXiv  Detail & Related papers  (2026-04-09T07:37:09Z)
• SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images [51.42466259821335]
  We present a preliminary exploration of applying SAM 3 to the remote sensing OVSS task without any training.<n>First, we implement a mask fusion strategy that combines the outputs from SAM 3's semantic segmentation head and the Transformer decoder.<n>Second, we utilize the presence score from the presence head to filter out categories that do not exist in the scene.
  arXiv  Detail & Related papers  (2025-12-09T15:42:28Z)
• VesSAM: Efficient Multi-Prompting for Segmenting Complex Vessel [68.24765319399286]
  We present VesSAM, a powerful and efficient framework tailored for 2D vessel segmentation.<n>VesSAM integrates (1) a convolutional adapter to enhance local texture features, (2) a multi-prompt encoder that fuses anatomical prompts, and (3) a lightweight mask decoder to reduce jagged artifacts.<n>VesSAM consistently outperforms state-of-the-art PEFT-based SAM variants by over 10% Dice and 13% IoU.
  arXiv  Detail & Related papers  (2025-11-02T15:47:05Z)
• Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentation [26.786890883280062]
  We introduce Trident, a training-free framework that first splices features extracted by CLIP and DINO from sub-images, then leverages SAM's encoder to create a correlation matrix for global aggregation. Trident achieves a significant improvement in the mIoU across eight benchmarks compared with the current SOTA, increasing from 44.4 to 48.6.Code.
  arXiv  Detail & Related papers  (2024-11-14T06:31:20Z)
• EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model [39.967352995143855]
  We introduce the Early Vision-language Fusion-based SAM (EVF-SAM)<n>EVF-SAM is a simple yet effective referring segmentation method which exploits multimodal prompts (i.e., image and text)<n>Experiments show that the proposed EVF-SAM based on BEIT-3 can obtain state-of-the-art performance on RefCOCO/+/g for referring expression segmentation.
  arXiv  Detail & Related papers  (2024-06-28T17:38:18Z)
• PosSAM: Panoptic Open-vocabulary Segment Anything [58.72494640363136]
  PosSAM is an open-vocabulary panoptic segmentation model that unifies the strengths of the Segment Anything Model (SAM) with the vision-native CLIP model in an end-to-end framework. We introduce a Mask-Aware Selective Ensembling (MASE) algorithm that adaptively enhances the quality of generated masks and boosts the performance of open-vocabulary classification during inference for each image.
  arXiv  Detail & Related papers  (2024-03-14T17:55:03Z)
• Stable Segment Anything Model [79.9005670886038]
  The Segment Anything Model (SAM) achieves remarkable promptable segmentation given high-quality prompts. This paper presents the first comprehensive analysis on SAM's segmentation stability across a diverse spectrum of prompt qualities. Our solution, termed Stable-SAM, offers several advantages: 1) improved SAM's segmentation stability across a wide range of prompt qualities, while 2) retaining SAM's powerful promptable segmentation efficiency and generality.
  arXiv  Detail & Related papers  (2023-11-27T12:51:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.