WOW-Seg: A Word-free Open World Segmentation Model

• URL: http://arxiv.org/abs/2605.16903v1
• Date: Sat, 16 May 2026 09:28:46 GMT
• Title: WOW-Seg: A Word-free Open World Segmentation Model
• Abstract summary: We propose WOW-Seg, a Word-free Open World model for segmenting and recognizing objects from open-set categories.<n>Specifically, WOW-Seg introduces a novel visual prompt module, Mask2Token, which transforms image masks into visual tokens.<n>We also construct an open world region recognition test benchmark: the Region Recognition dataset.
• Score: 54.80135620139728
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open world image segmentation aims to achieve precise segmentation and semantic understanding of targets within images by addressing the infinitely open set of object categories encountered in the real world. However, traditional closed-set segmentation approaches struggle to adapt to complex open world scenarios, while foundation segmentation models such as SAM exhibit notable discrepancies between their strong segmentation capabilities and relatively weaker semantic understanding. To bridge these discrepancies, we propose WOW-Seg, a Word-free Open World Segmentation model for segmenting and recognizing objects from open-set categories. Specifically, WOW-Seg introduces a novel visual prompt module, Mask2Token, which transforms image masks into visual tokens and ensures their alignment with the VLLM feature space. Moreover, we introduce the Cascade Attention Mask to decouple information across different instances. This approach mitigates inter-instance interference, leading to a significant improvement in model performance. We further construct an open world region recognition test benchmark: the Region Recognition Dataset (RR-7K). With 7,662 classes, it represents the most extensive category-rich region recognition dataset to date. WOW-Seg attains strong results on the LVIS dataset, achieving a semantic similarity of 89.7 and a semantic IoU of 82.4. This performance surpasses the previous SOTA while using only one-eighth the parameter count. These results underscore the strong open world generalization capabilities of WOW-Seg. The code and related resources are available at https://github.com/AAwcAA/WOW-Seg-Meta.

Related papers

• LoGoSeg: Integrating Local and Global Features for Open-Vocabulary Semantic Segmentation [12.192429756057132]
  Open-vocabulary semantic segmentation (OVSS) extends traditional closed-set segmentation by enabling pixel-wise annotation for both seen and unseen categories.<n>LoGoSeg integrates three key innovations: (i) an object existence prior that dynamically weights relevant categories through global image-text similarity, effectively reducing hallucinations; (ii) a region-aware alignment module that establishes precise region-level visual-textual correspondences; and (iii) a dual-stream fusion mechanism that optimally combines local structural information with global semantic context.
  arXiv  Detail & Related papers  (2026-02-05T12:03:11Z)
• Segment and Matte Anything in a Unified Model [5.8874968768571625]
  Segment Anything (SAM) has recently pushed the boundaries of segmentation by demonstrating zero-shot generalization and flexible prompting.<n>We introduce Segment And Matte Anything (SAMA), a lightweight extension of SAM that delivers high-quality interactive image segmentation and matting.
  arXiv  Detail & Related papers  (2026-01-17T19:43:10Z)
• OpenWorldSAM: Extending SAM2 for Universal Image Segmentation with Language Prompts [20.327695503392274]
  We present OpenWorldSAM, a framework that extends the prompt-driven Segment Anything Model v2 (SAM2) to open-vocabulary scenarios.<n>OpenWorldSAM supports a diverse range of prompts, including category-level and sentence-level language descriptions.<n>We enhance the model's spatial understanding through novel positional tie-breaker embeddings and cross-attention layers.
  arXiv  Detail & Related papers  (2025-07-07T19:16:22Z)
• RemoteSAM: Towards Segment Anything for Earth Observation [29.707796048411705]
  We aim to develop a robust yet flexible visual foundation model for Earth observation.<n>It should possess strong capabilities in recognizing and localizing diverse visual targets.<n>We present RemoteSAM, a foundation model that establishes new SoTA on several earth observation perception benchmarks.
  arXiv  Detail & Related papers  (2025-05-23T15:27:57Z)
• PosSAM: Panoptic Open-vocabulary Segment Anything [58.72494640363136]
  PosSAM is an open-vocabulary panoptic segmentation model that unifies the strengths of the Segment Anything Model (SAM) with the vision-native CLIP model in an end-to-end framework. We introduce a Mask-Aware Selective Ensembling (MASE) algorithm that adaptively enhances the quality of generated masks and boosts the performance of open-vocabulary classification during inference for each image.
  arXiv  Detail & Related papers  (2024-03-14T17:55:03Z)
• Grounding Everything: Emerging Localization Properties in Vision-Language Transformers [51.260510447308306]
  We show that pretrained vision-language (VL) models allow for zero-shot open-vocabulary object localization without any fine-tuning. We propose a Grounding Everything Module (GEM) that generalizes the idea of value-value attention introduced by CLIPSurgery to a self-self attention path. We evaluate the proposed GEM framework on various benchmark tasks and datasets for semantic segmentation.
  arXiv  Detail & Related papers  (2023-12-01T19:06:12Z)
• A Lightweight Clustering Framework for Unsupervised Semantic Segmentation [28.907274978550493]
  Unsupervised semantic segmentation aims to categorize each pixel in an image into a corresponding class without the use of annotated data. We propose a lightweight clustering framework for unsupervised semantic segmentation. Our framework achieves state-of-the-art results on PASCAL VOC and MS COCO datasets.
  arXiv  Detail & Related papers  (2023-11-30T15:33:42Z)
• Semantic-SAM: Segment and Recognize Anything at Any Granularity [83.64686655044765]
  We introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. We consolidate multiple datasets across three granularities and introduce decoupled classification for objects and parts. For the multi-granularity capability, we propose a multi-choice learning scheme during training, enabling each click to generate masks at multiple levels.
  arXiv  Detail & Related papers  (2023-07-10T17:59:40Z)
• Towards Universal Vision-language Omni-supervised Segmentation [72.31277932442988]
  We present Vision-Language Omni-Supervised (VLOSS) to treat open-world segmentation tasks as proposal classification. We leverage omni-supervised data (i.e., panoptic segmentation data, object detection data, and image-text pairs data) into training, thus enriching the open-world segmentation ability. With fewer parameters, our VLOSS with Swin-Tiny surpasses MaskCLIP by 2% in terms of mask AP on LVIS v1 dataset.
  arXiv  Detail & Related papers  (2023-03-12T02:57:53Z)
• Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity [59.1823948436411]
  We propose a novel approach for mask proposals, Generic Grouping Networks (GGNs) Our approach combines a local measure of pixel affinity with instance-level mask supervision, producing a training regimen designed to make the model as generic as the data diversity allows.
  arXiv  Detail & Related papers  (2022-04-12T22:37:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.