Learning to Label: A Reinforced Self-Evolving Framework for Semi-supervised Referring Expression Segmentation

• URL: http://arxiv.org/abs/2605.28239v1
• Date: Wed, 27 May 2026 09:54:26 GMT
• Title: Learning to Label: A Reinforced Self-Evolving Framework for Semi-supervised Referring Expression Segmentation
• Abstract summary: Learning to Label casts pseudo-label construction as a learnable decision-making process.<n>To ensure stable learning, reinforced pseudo-label selection is formulated as an exploratory decision process.<n>Experiments on RefCOCO, RefCO+, and RefCOCOg demonstrate improvements over existing methods.
• Score: 34.401271440453044
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semi-supervised referring expression segmentation (SS-RES) aims to achieve precise pixel-level language grounding under limited annotation, yet suffers from limited supervision and unreliable pseudo-labels when exploiting unlabeled image-text pairs. In this work, we propose Learning to Label, a reinforced self-evolving framework (L2L) that casts pseudo-label construction as a learnable decision-making process. To build foundational understanding, we leverage a multimodal large language model to extract semantic-spatial priors, which are instantiated as initial soft segmentation proposals and elevated, together with textual cues, into learnable guidance signals that condition a hierarchical segmentation network. To ensure stable learning, reinforced pseudo-label selection is formulated as an exploratory decision process that adaptively rewards high-utility pixel-level supervision based on multimodal priors and model predictions. This reinforced self-evolving loop enables joint optimization of the segmentation model and pseudo-labels, progressively enhancing label reliability under sparse supervision. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate improvements over existing methods, validating its effectiveness and generalization.

Related papers

• Learning a Semantic Calibration Network for Open-Vocabulary Semantic Segmentation [5.122331812021513]
  Open-Vocabulary (OVS) extends the segmentation task from a fixed set to an open set.<n>We propose a novel Semantic Network (SCN) for open-vocabulary semantic segmentation.<n>The proposed method achieves significant performance improvements compared to state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2026-06-06T06:42:03Z)
• Toward Stable Semi-Supervised Remote Sensing Segmentation via Co-Guidance and Co-Fusion [31.189038928192648]
  Co2S is a semi-supervised RS segmentation framework that fuses priors from vision-language models and self-supervised models.<n>An explicit-implicit semantic co-guidance mechanism is introduced that utilizes text embeddings and learnable queries.<n>Experiments on six popular datasets demonstrate the superiority of the proposed method.
  arXiv  Detail & Related papers  (2025-12-28T18:24:19Z)
• AURORA: Augmented Understanding via Structured Reasoning and Reinforcement Learning for Reference Audio-Visual Segmentation [113.75682363364004]
  AURORA is a framework designed to enhance genuine reasoning and language comprehension in reference audio-visual segmentation.<n>AURORA achieves state-of-the-art performance on Ref-AVS benchmarks and generalizes effectively to unreferenced segmentation.
  arXiv  Detail & Related papers  (2025-08-04T07:47:38Z)
• ConformalSAM: Unlocking the Potential of Foundational Segmentation Models in Semi-Supervised Semantic Segmentation with Conformal Prediction [57.930531826380836]
  This work explores whether a foundational segmentation model can address label scarcity in the pixel-level vision task as an annotator for unlabeled images.<n>We propose ConformalSAM, a novel SSSS framework which first calibrates the foundation model using the target domain's labeled data and then filters out unreliable pixel labels of unlabeled data.
  arXiv  Detail & Related papers  (2025-07-21T17:02:57Z)
• Semantic-Aligned Learning with Collaborative Refinement for Unsupervised VI-ReID [82.12123628480371]
  Unsupervised person re-identification (USL-VI-ReID) seeks to match pedestrian images of the same individual across different modalities without human annotations for model learning.<n>Previous methods unify pseudo-labels of cross-modality images through label association algorithms and then design contrastive learning framework for global feature learning.<n>We propose a Semantic-Aligned Learning with Collaborative Refinement (SALCR) framework, which builds up objective for specific fine-grained patterns emphasized by each modality.
  arXiv  Detail & Related papers  (2025-04-27T13:58:12Z)
• IGL-DT: Iterative Global-Local Feature Learning with Dual-Teacher Semantic Segmentation Framework under Limited Annotation Scheme [4.247494613524502]
  Semi-Supervised Semantic (SSSS) aims to improve segmentation accuracy by leveraging a small set of labeled images alongside a larger pool of unlabeled data.<n>We propose a novel tri-branch semi-supervised segmentation framework incorporating a dual-teacher strategy, named IGL-DT.<n>Our approach employs SwinUnet for high-level semantic guidance through Global Context Learning and ResUnet for detailed feature refinement via Local Regional Learning.
  arXiv  Detail & Related papers  (2025-04-14T01:51:29Z)
• Uncertainty-Participation Context Consistency Learning for Semi-supervised Semantic Segmentation [9.546065701435532]
  Semi-supervised semantic segmentation has attracted considerable attention for its ability to mitigate the reliance on extensive labeled data.<n>This paper proposes the Uncertainty-participation Context Consistency Learning (UCCL) method to explore richer supervisory signals.
  arXiv  Detail & Related papers  (2024-12-23T06:49:59Z)
• Semi-supervised Semantic Segmentation Meets Masked Modeling:Fine-grained Locality Learning Matters in Consistency Regularization [31.333862320143968]
  Semi-supervised semantic segmentation aims to utilize limited labeled images and abundant unlabeled images to achieve label-efficient learning. We propose a novel framework called textttMaskMatch, which enables fine-grained locality learning to achieve better dense segmentation.
  arXiv  Detail & Related papers  (2023-12-14T03:28:53Z)
• Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation [89.41179071022121]
  Self-training is a prevailing approach in cross-domain semantic segmentation. We propose a novel approach called Semantic Connectivity-driven pseudo-labeling. This approach formulates pseudo-labels at the connectivity level and thus can facilitate learning structured and low-noise semantics.
  arXiv  Detail & Related papers  (2023-12-11T12:29:51Z)
• Scalable Learning of Latent Language Structure With Logical Offline Cycle Consistency [71.42261918225773]
  Conceptually, LOCCO can be viewed as a form of self-learning where the semantic being trained is used to generate annotations for unlabeled text. As an added bonus, the annotations produced by LOCCO can be trivially repurposed to train a neural text generation model.
  arXiv  Detail & Related papers  (2023-05-31T16:47:20Z)
• Neighbour Consistency Guided Pseudo-Label Refinement for Unsupervised Person Re-Identification [80.98291772215154]
  Unsupervised person re-identification (ReID) aims at learning discriminative identity features for person retrieval without any annotations. Recent advances accomplish this task by leveraging clustering-based pseudo labels. We propose a Neighbour Consistency guided Pseudo Label Refinement framework.
  arXiv  Detail & Related papers  (2022-11-30T09:39:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.