Training-Free Generalized Few-Shot Segmentation through Open-Vocabulary Semantic Arbitration

• URL: http://arxiv.org/abs/2606.09474v1
• Date: Mon, 08 Jun 2026 13:30:22 GMT
• Title: Training-Free Generalized Few-Shot Segmentation through Open-Vocabulary Semantic Arbitration
• Abstract summary: Generalized Few-Shot Semantic (GFSS) has traditionally been approached as a representation-learning problem.<n>Recent foundation models, however, already exhibit strong open-vocabulary recognition and segmentation capabilities.<n>We show that Open-V can be solved through inference-time coordination of frozen semantic priors rather than adaptation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalized Few-Shot Semantic Segmentation (GFSS) has traditionally been approached as a representation-learning problem, requiring task-specific adaptation to incorporate novel classes from limited support examples. Recent foundation models, however, already exhibit strong open-vocabulary recognition and segmentation capabilities, raising a different question: can GFSS be solved through inference-time coordination of frozen semantic priors rather than parameter adaptation? We answer this question with Open-V, a training-free GFSS framework that combines Segment Anything (SAM3) Promptable Concept Segmentation (PCS) with a K-shot CLIP support centroid through calibrated per-pixel semantic arbitration. OpenV introduces no trainable components and supports arbitrary semantic categories at inference time. Beyond segmentation performance, our study contributes three broader findings. First, we show that support information can be incorporated through inference-time semantic grounding, and that its contribution increases as foundation-model text priors weaken on label-disjoint vocabularies. Second, we identify a reproducibility confound in foundationmodel segmentation, demonstrating that preprocessing and evaluation-space mismatches can silently distort reported performance. Finally, we validate Open-V across PASCAL5i, COCO-20i, and ADE-OW, showing that training-free coordination of foundation-model priors generalizes across both conventional GFSS and open-vocabulary evaluation settings. On PASCAL-5i (1-shot), Open-V attains base/novel/harmonic mIoU of 78.4/77.5/77.9, without GFSS-specific training surpassing the strongest trained baseline by +17.7 HM.

Related papers

• Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentation [38.16802763051431]
  We propose CLIPtrase, a training-free semantic segmentation strategy. It enhances local feature awareness through recalibrated self-correlation among patches. Experiments show that we are 22.3% ahead of CLIP on average on 9 segmentation benchmarks.
  arXiv  Detail & Related papers  (2024-07-11T08:12:16Z)
• Pay Attention to Your Neighbours: Training-Free Open-Vocabulary Semantic Segmentation [19.20874993309959]
  vision-language foundation models, such as CLIP, have showcased remarkable effectiveness in numerous zero-shot image-level tasks. We propose a baseline for training-free OVSS, termed Neighbour-Aware CLIP (NACLIP) Our method enforces localization of patches in the self-attention of CLIP's vision transformer which, despite being crucial for dense prediction tasks, has been overlooked in the OVSS literature.
  arXiv  Detail & Related papers  (2024-04-12T01:08:04Z)
• Open-Vocabulary Segmentation with Semantic-Assisted Calibration [68.41025728960176]
  We study open-vocabulary segmentation (OVS) through calibrating in-vocabulary and domain-biased embedding space with contextual prior of CLIP.<n>We present a Semantic-assisted CAlibration Network (SCAN) to achieve state-of-the-art performance on open-vocabulary segmentation benchmarks.
  arXiv  Detail & Related papers  (2023-12-07T07:00:09Z)
• Prototypical Kernel Learning and Open-set Foreground Perception for Generalized Few-shot Semantic Segmentation [7.707161030443157]
  Generalized Few-shot Semantic (GFSS) extends Few-shot Semantic aggregation to segment unseen classes and seen classes during evaluation. We address the aforementioned problems by jointing the prototypical kernel learning and open-set perception. In addition, a foreground contextual perception module cooperating with conditional bias based inference is adopted to perform class-agnostic as well as open-set foreground detection.
  arXiv  Detail & Related papers  (2023-08-09T13:38:52Z)
• Self-regulating Prompts: Foundational Model Adaptation without Forgetting [112.66832145320434]
  We introduce a self-regularization framework for prompting called PromptSRC. PromptSRC guides the prompts to optimize for both task-specific and task-agnostic general representations.
  arXiv  Detail & Related papers  (2023-07-13T17:59:35Z)
• Global Knowledge Calibration for Fast Open-Vocabulary Segmentation [124.74256749281625]
  We introduce a text diversification strategy that generates a set of synonyms for each training category. We also employ a text-guided knowledge distillation method to preserve the generalizable knowledge of CLIP. Our proposed model achieves robust generalization performance across various datasets.
  arXiv  Detail & Related papers  (2023-03-16T09:51:41Z)
• Learning What Not to Segment: A New Perspective on Few-Shot Segmentation [63.910211095033596]
  Recently few-shot segmentation (FSS) has been extensively developed. This paper proposes a fresh and straightforward insight to alleviate the problem. In light of the unique nature of the proposed approach, we also extend it to a more realistic but challenging setting.
  arXiv  Detail & Related papers  (2022-03-15T03:08:27Z)
• Generalized Few-shot Semantic Segmentation [68.69434831359669]
  We introduce a new benchmark called Generalized Few-Shot Semantic (GFS-Seg) to analyze the ability of simultaneously segmenting the novel categories. It is the first study showing that previous representative state-of-the-art generalizations fall short in GFS-Seg. We propose the Context-Aware Prototype Learning (CAPL) that significantly improves performance by 1) leveraging the co-occurrence prior knowledge from support samples, and 2) dynamically enriching contextual information to the conditioned, on the content of each query image.
  arXiv  Detail & Related papers  (2020-10-11T10:13:21Z)
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation [64.91560451900125]
  State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results. Few-shot segmentation is proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples. Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information.
  arXiv  Detail & Related papers  (2020-08-04T10:41:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.