ConceptSeg-R1: Segment Any Concept via Meta-Reinforcement Learning

• URL: http://arxiv.org/abs/2605.20385v1
• Date: Tue, 19 May 2026 18:35:38 GMT
• Title: ConceptSeg-R1: Segment Any Concept via Meta-Reinforcement Learning
• Abstract summary: Recent progress in promptable segmentation has shifted visual perception from object-level localization toward concept-level understanding.<n>We formalize concept segmentation through a three-level taxonomy consisting of context-independent (CI), context-dependent (CD), and context-reasoning (CR) concepts.<n>We propose ConceptSeg-R1, a unified framework that reformulates concept segmentation as rule-induced concept grounding.
• Score: 134.61875033137514
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in promptable segmentation has shifted visual perception from object-level localization toward concept-level understanding. However, the notion of a concept remains under-specified, making it unclear whether current methods truly generalize beyond category recognition. In this work, we formalize generalized concept segmentation through a three-level taxonomy consisting of context-independent (CI), context-dependent (CD), and context-reasoning (CR) concepts, which reveals a clear capability gap across increasing levels of cognitive complexity. To address this challenge, we propose ConceptSeg-R1, a unified framework that reformulates concept segmentation as rule-induced concept grounding. At the core of our method is Meta-GRPO, a meta-reinforcement learning mechanism that learns transferable task rules from visual demonstrations and verifies them through proxy reasoning. The inferred reasoning states are then translated into segmentation-ready concept prompts via a lightweight concept translation module, enabling deductive application to target images. A shortcut routing strategy further preserves the native efficiency of segmentation models on simple cases. To systematically evaluate generalized concept segmentation, we conduct extensive experiments across diverse CI, CD, and CR concept segmentation benchmarks spanning natural, industrial, medical and reasoning-intensive domains. Without bells and whistles, ConceptSeg-R1 achieves strong performance across the full concept hierarchy while maintaining the native capability of promptable segmentation backbones. As an initial step toward segmenting any concept, we hope ConceptSeg-R1 can serve as a practical baseline for advancing segmentation from object-level prediction toward concept-level understanding.

Related papers

• Explaining CLIP Zero-shot Predictions Through Concepts [54.05282304471016]
  We introduce EZPC to explain CLIP's zero-shot predictions through human-understandable concepts.<n>Our method projects CLIP's joint image-text embeddings into a concept space learned from language descriptions.<n>Our approach maintains CLIP's strong zero-shot classification accuracy while providing meaningful concept-level explanations.
  arXiv  Detail & Related papers  (2026-03-30T09:31:33Z)
• SCoCCA: Multi-modal Sparse Concept Decomposition via Canonical Correlation Analysis [7.4746628878102195]
  We introduce Concept CCA (CoCCA), a framework that aligns cross-modal embeddings while enabling interpretable concept decomposition.<n>We propose Sparse Concept CCA (SCoCCA), which enforces sparsity to produce more disentangled and discriminative concepts.<n>Our approach generalizes concept-based explanations to multi-modal embeddings and facilitates state-of-the-art performance in concept discovery.
  arXiv  Detail & Related papers  (2026-03-14T10:45:59Z)
• Intrinsic Concept Extraction Based on Compositional Interpretability [34.98614596462498]
  This paper introduces a new task called Compositional and Interpretable Intrinsic Concept Extraction (CI-ICE)<n>The CI-ICE task aims to leverage diffusion-based text-to-image models to extract composable object-level and attribute-level concepts from a single image.<n>Our method demonstrates outstanding performance in extracting compositionally interpretable intrinsic concepts from a single image.
  arXiv  Detail & Related papers  (2026-03-12T10:56:05Z)
• Concept Component Analysis: A Principled Approach for Concept Extraction in LLMs [51.378834857406325]
  Mechanistic interpretability seeks to mitigate the issues through extracts from large language models.<n>Sparse autoencoders (SAEs) have emerged as a popular approach for extracting interpretable and monosemantic concepts.<n>We show that SAEs suffer from a fundamental theoretical ambiguity: the well-defined correspondence between LLM representations and human-interpretable concepts remains unclear.
  arXiv  Detail & Related papers  (2026-01-28T09:27:05Z)
• FaCT: Faithful Concept Traces for Explaining Neural Network Decisions [56.796533084868884]
  Deep networks have shown remarkable performance across a wide range of tasks, yet getting a global concept-level understanding of how they function remains a key challenge.<n>We put emphasis on the faithfulness of concept-based explanations and propose a new model with model-inherent mechanistic concept-explanations.<n>Our concepts are shared across classes and, from any layer, their contribution to the logit and their input-visualization can be faithfully traced.
  arXiv  Detail & Related papers  (2025-10-29T13:35:46Z)
• Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations [19.50321703079894]
  We introduce an effective circuit discovery method, called Granular Concept Circuit (GCC), in which each circuit represents a concept relevant to a given query.<n>By automatically discovering multiple circuits, each capturing specific concepts within that query, our approach offers a profound, concept-wise interpretation of models.<n>We validate the versatility and effectiveness of GCCs across various deep image classification models.
  arXiv  Detail & Related papers  (2025-08-03T11:45:38Z)
• SeC: Advancing Complex Video Object Segmentation via Progressive Concept Construction [65.15449703659772]
  Video Object (VOS) is a core task in computer vision, requiring models to track and segment target objects across video frames.<n>We propose Segment Concept (SeC), a concept-driven segmentation framework that shifts from conventional feature matching to the progressive construction and utilization of high-level, object-centric representations.<n>SeC achieves an 11.8-point improvement over SAM SeCVOS, establishing a new state-of-the-art concept-aware video object segmentation.
  arXiv  Detail & Related papers  (2025-07-21T17:59:02Z)
• Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces [1.0923877073891446]
  Insights into the learned latent representations are imperative for verifying deep neural networks (DNNs) in computer vision tasks.<n>We propose a novel local concept analysis framework to allow exploration of learned concept distributions.<n>Despite its context sensitivity, our method's concept segmentation performance is competitive to global baselines.
  arXiv  Detail & Related papers  (2023-11-24T12:22:00Z)
• Coarse-to-Fine Concept Bottleneck Models [9.910980079138206]
  This work targets ante hoc interpretability, and specifically Concept Bottleneck Models (CBMs) Our goal is to design a framework that admits a highly interpretable decision making process with respect to human understandable concepts, on two levels of granularity. Within this framework, concept information does not solely rely on the similarity between the whole image and general unstructured concepts; instead, we introduce the notion of concept hierarchy to uncover and exploit more granular concept information residing in patch-specific regions of the image scene.
  arXiv  Detail & Related papers  (2023-10-03T14:57:31Z)
• Concept Gradient: Concept-based Interpretation Without Linear Assumption [77.96338722483226]
  Concept Activation Vector (CAV) relies on learning a linear relation between some latent representation of a given model and concepts. We proposed Concept Gradient (CG), extending concept-based interpretation beyond linear concept functions. We demonstrated CG outperforms CAV in both toy examples and real world datasets.
  arXiv  Detail & Related papers  (2022-08-31T17:06:46Z)
• GlanceNets: Interpretabile, Leak-proof Concept-based Models [23.7625973884849]
  Concept-based models (CBMs) combine high-performance and interpretability by acquiring and reasoning with a vocabulary of high-level concepts. We provide a clear definition of interpretability in terms of alignment between the model's representation and an underlying data generation process. We introduce GlanceNets, a new CBM that exploits techniques from disentangled representation learning and open-set recognition to achieve alignment.
  arXiv  Detail & Related papers  (2022-05-31T08:53:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.