Toward Robust In-Context Segmentation via Concept Guidance

• URL: http://arxiv.org/abs/2606.28149v1
• Date: Fri, 26 Jun 2026 14:46:38 GMT
• Title: Toward Robust In-Context Segmentation via Concept Guidance
• Abstract summary: In-context segmentation (ICS) requires a model to segment target regions in a query image using only a few reference images and their corresponding masks, without updating any parameters.<n>We introduce Concept-Guided In-Context (CG-ICS), which performs segmentation by extracting high-level semantic concepts from references.<n>Experiments on standard ICS benchmarks demonstrate that CG-ICS achieves state-of-the-art accuracy and substantially improves robustness, yielding a more reliable ICS system with significantly reduced variance across diverse reference choices.
• Score: 72.30282710763969
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In-context segmentation (ICS) requires a model to segment target regions in a query image using only a few reference images and their corresponding masks, without updating any parameters. Despite recent progress, prior ICS studies have largely overlooked a critical aspect: system robustness, ie, whether the model can produce stable segmentation results for the same query under different references. In this work, we revisit ICS from the robustness perspective and introduce a novel paradigm, Concept-Guided In-Context Segmentation (CG-ICS), which performs segmentation by extracting high-level semantic concepts from references rather than relying solely on low-level visual matching. Specifically, CG-ICS introduces a concept reasoning module that uses an MLLM to propose candidates and a SAM3-driven scoring function with tree-search refinement to select reliable textual concepts, together with a parallel visual exemplar route that provides query-side spatial grounding via a simple context construction. Both the textual concept and the visual exemplar are then used to activate the segmentation capability of a frozen SAM3 backbone. Extensive experiments on standard ICS benchmarks demonstrate that CG-ICS not only achieves state-of-the-art accuracy but also substantially improves robustness, yielding a more reliable ICS system with significantly reduced variance across diverse reference choices.

Related papers

• Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification [21.7465671470498]
  Referring segmentation aims to segment the target objects in images or videos based on the textual query.<n>We propose IC-Seg, a novel agentic framework that proactively clarifies user intent through multi-turn conversation.<n>We also introduce Hi-GRPO, a new hierarchical optimization strategy that injects dense and informative supervision signals at the trajectory, turn, and step levels.
  arXiv  Detail & Related papers  (2026-05-17T16:30:44Z)
• RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentation [48.30592530624143]
  Video Reasoning aims to segment target objects in videos based on implicit instructions that convey human intent and temporal logic.<n>Existing MLLM-based methods predict masks with a [SEG] token after selecting frames via simple sampling or an auxiliary MLLM.<n>We introduce RCoT-Seg, a video-of-thought framework that factorizes VRS into temporal video (TVR) and target perception (KTP)
  arXiv  Detail & Related papers  (2026-05-08T06:39:53Z)
• INSID3: Training-Free In-Context Segmentation with DINOv3 [63.961377087673476]
  INSID3 is a training-free approach that segments concepts at varying granularities only from frozen DINOv3 features.<n>It achieves state-of-the-art results across one-shot semantic, part, and personalized segmentation, outperforming previous work by +7.5 % mIoU.
  arXiv  Detail & Related papers  (2026-03-30T14:16:37Z)
• CSMCIR: CoT-Enhanced Symmetric Alignment with Memory Bank for Composed Image Retrieval [54.15776146365823]
  Composed Image Retrieval (CIR) enables users to search for target images using both a reference image and manipulation text.<n>We propose CSMCIR, a unified representation framework that achieves efficient query-target alignment through three synergistic components.
  arXiv  Detail & Related papers  (2026-01-07T09:21:38Z)
• Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design [8.993770750003673]
  Few-Shot Semantic (FSS) models achieve strong performance in segmenting novel classes with minimal labeled examples.<n>This paper introduces the first dedicated method for interpreting matching-based FSS models.<n>Our Affinity Explainer approach extracts attribution maps that highlight which pixels in support images contribute most to query segmentation predictions.
  arXiv  Detail & Related papers  (2025-11-22T19:22:10Z)
• SeC: Advancing Complex Video Object Segmentation via Progressive Concept Construction [65.15449703659772]
  Video Object (VOS) is a core task in computer vision, requiring models to track and segment target objects across video frames.<n>We propose Segment Concept (SeC), a concept-driven segmentation framework that shifts from conventional feature matching to the progressive construction and utilization of high-level, object-centric representations.<n>SeC achieves an 11.8-point improvement over SAM SeCVOS, establishing a new state-of-the-art concept-aware video object segmentation.
  arXiv  Detail & Related papers  (2025-07-21T17:59:02Z)
• Visual Prompt Selection for In-Context Learning Segmentation [77.15684360470152]
  In this paper, we focus on rethinking and improving the example selection strategy. We first demonstrate that ICL-based segmentation models are sensitive to different contexts. Furthermore, empirical evidence indicates that the diversity of contextual prompts plays a crucial role in guiding segmentation.
  arXiv  Detail & Related papers  (2024-07-14T15:02:54Z)
• SEGIC: Unleashing the Emergent Correspondence for In-Context Segmentation [87.18373801829314]
  In-context segmentation aims at segmenting novel images using a few labeled example images, termed as "in-context examples" We propose SEGIC, an end-to-end segment-in-context framework built upon a single vision foundation model (VFM) SEGIC is a straightforward yet effective approach that yields state-of-the-art performance on one-shot segmentation benchmarks.
  arXiv  Detail & Related papers  (2023-11-24T18:59:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.