Prompt Group-Aware Training for Robust Text-Guided Nuclei Segmentation

• URL: http://arxiv.org/abs/2603.06384v1
• Date: Fri, 06 Mar 2026 15:34:08 GMT
• Title: Prompt Group-Aware Training for Robust Text-Guided Nuclei Segmentation
• Abstract summary: We reformulate prompt sensitivity as a group-wise consistency problem.<n>We show that our method improves Dice by an average of 2.16 points on six zero-shot cross-dataset tasks.
• Score: 6.329891529118214
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models such as Segment Anything Model 3 (SAM3) enable flexible text-guided medical image segmentation, yet their predictions remain highly sensitive to prompt formulation. Even semantically equivalent descriptions can yield inconsistent masks, limiting reliability in clinical and pathology workflows. We reformulate prompt sensitivity as a group-wise consistency problem. Semantically related prompts are organized into \emph{prompt groups} sharing the same ground-truth mask, and a prompt group-aware training framework is introduced for robust text-guided nuclei segmentation. The approach combines (i) a quality-guided group regularization that leverages segmentation loss as an implicit ranking signal, and (ii) a logit-level consistency constraint with a stop-gradient strategy to align predictions within each group. The method requires no architectural modification and leaves inference unchanged. Extensive experiments on multi-dataset nuclei benchmarks show consistent gains under textual prompting and markedly reduced performance variance across prompt quality levels. On six zero-shot cross-dataset tasks, our method improves Dice by an average of 2.16 points. These results demonstrate improved robustness and generalization for vision-language segmentation in computational pathology.

Related papers

• Imputation Meets Clustering: Exploiting Latent Subgroup Structure for Missing Data Recovery [52.29866804948133]
  We propose CAGI (Cluster-Aware Generative Imputation), a framework that reformulates clustering and imputation as a mutually reinforcing co-optimization process.<n>Experiments on 14 benchmark datasets with 15 representative baselines demonstrate the superiority of CAGI.
  arXiv  Detail & Related papers  (2026-07-08T02:50:02Z)
• STaR-DRO: Stateful Tsallis Reweighting for Group-Robust Structured Prediction [0.20165644958637055]
  We present a two-part framework for controllable inference and robust fine-tuning.<n>First, we introduce a task-agnostic prompting strategy to address format drift, label ambiguity, evidence hallucination, and metadata-conditioned confusion.<n>Second, we introduce STaR-DRO, a stateful robust optimization method for group heterogeneity.
  arXiv  Detail & Related papers  (2026-04-09T21:58:42Z)
• Structured Prompt Optimization for Few-Shot Text Classification via Semantic Alignment in Latent Space [3.66782592128973]
  This study addresses the issues of semantic entanglement, unclear label structure, and insufficient feature representation in few-shot text classification.<n>It proposes an optimization framework based on structured prompts to enhance semantic understanding and task adaptation under low-resource conditions.<n> Experimental results show that the proposed structured prompt optimization framework effectively alleviates semantic conflicts and label ambiguity in few-shot text classification.
  arXiv  Detail & Related papers  (2026-02-27T07:31:16Z)
• AgenticSum: An Agentic Inference-Time Framework for Faithful Clinical Text Summarization [6.99563009617414]
  We present AgenticSum, an inference-time framework that separates context selection, generation, verification, and targeted correction to reduce hallucinated content.<n>We evaluate AgenticSum on two public datasets, using reference-based metrics, LLM-as-a-judge assessment, and human evaluation.<n>Our results indicate that structured, agentic design with targeted correction offers an effective inference time solution to improve clinical note summarization.
  arXiv  Detail & Related papers  (2026-02-23T16:49:37Z)
• Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection [58.419940585826744]
  We introduce FairOPT, an algorithm for group-specific threshold optimization for probabilistic AI-text detectors.<n>We partitioned data into subgroups based on attributes (e.g., text length and writing style) and implemented FairOPT to learn decision thresholds for each group to reduce discrepancy.<n>Our framework paves the way for more robust classification in AI-generated content detection via post-processing.
  arXiv  Detail & Related papers  (2025-02-06T21:58:48Z)
• SemSim: Revisiting Weak-to-Strong Consistency from a Semantic Similarity Perspective for Semi-supervised Medical Image Segmentation [18.223854197580145]
  Semi-supervised learning (SSL) for medical image segmentation is a challenging yet highly practical task. We propose a novel framework based on FixMatch, named SemSim, powered by two appealing designs from semantic similarity perspective. We show that SemSim yields consistent improvements over the state-of-the-art methods across three public segmentation benchmarks.
  arXiv  Detail & Related papers  (2024-10-17T12:31:37Z)
• Multi-Grained Cross-modal Alignment for Learning Open-vocabulary Semantic Segmentation from Text Supervision [23.931443799102663]
  We introduce a Multi-Grained Cross-modal Alignment (MGCA) framework to bridge the granularity gap without any dense annotations. Specifically, MGCA constructs pseudo multi-granular semantic correspondences upon image-text pairs. Our method achieves significant advancements over state-of-the-art methods, demonstrating its effectiveness and efficiency.
  arXiv  Detail & Related papers  (2024-03-06T13:43:36Z)
• Sequential Visual and Semantic Consistency for Semi-supervised Text Recognition [56.968108142307976]
  Scene text recognition (STR) is a challenging task that requires large-scale annotated data for training. Most existing STR methods resort to synthetic data, which may introduce domain discrepancy and degrade the performance of STR models. This paper proposes a novel semi-supervised learning method for STR that incorporates word-level consistency regularization from both visual and semantic aspects.
  arXiv  Detail & Related papers  (2024-02-24T13:00:54Z)
• Uncovering Prototypical Knowledge for Weakly Open-Vocabulary Semantic Segmentation [59.37587762543934]
  This paper studies the problem of weakly open-vocabulary semantic segmentation (WOVSS) Existing methods suffer from a granularity inconsistency regarding the usage of group tokens. We propose the prototypical guidance network (PGSeg) that incorporates multi-modal regularization.
  arXiv  Detail & Related papers  (2023-10-29T13:18:00Z)
• Associating Spatially-Consistent Grouping with Text-supervised Semantic Segmentation [117.36746226803993]
  We introduce self-supervised spatially-consistent grouping with text-supervised semantic segmentation. Considering the part-like grouped results, we further adapt a text-supervised model from image-level to region-level recognition. Our method achieves 59.2% mIoU and 32.4% mIoU on Pascal VOC and Pascal Context benchmarks.
  arXiv  Detail & Related papers  (2023-04-03T16:24:39Z)
• Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
  In this paper, contextual hints are exploited via learning a context-aware classifier. Our method is model-agnostic and can be easily applied to generic segmentation models. With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
  arXiv  Detail & Related papers  (2023-03-21T07:00:35Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.