Holistic Optimal Label Selection for Robust Prompt Learning under Partial Labels

• URL: http://arxiv.org/abs/2604.06614v1
• Date: Wed, 08 Apr 2026 02:49:19 GMT
• Title: Holistic Optimal Label Selection for Robust Prompt Learning under Partial Labels
• Abstract summary: Holistic Optimal Label Selection (HopS) is a parameter-efficient approach for adapting large pre-trained vision-language models to downstream tasks.<n>We show that HopS consistently improves performance under partial supervision and outperforms all baselines.
• Score: 57.91658393667468
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prompt learning has gained significant attention as a parameter-efficient approach for adapting large pre-trained vision-language models to downstream tasks. However, when only partial labels are available, its performance is often limited by label ambiguity and insufficient supervisory information. To address this issue, we propose Holistic Optimal Label Selection (HopS), leveraging the generalization ability of pre-trained feature encoders through two complementary strategies. First, we design a local density-based filter that selects the top frequent labels from the nearest neighbors' candidate sets and uses the softmax scores to identify the most plausible label, capturing structural regularities in the feature space. Second, we introduce a global selection objective based on optimal transport that maps the uniform sampling distribution to the candidate label distributions across a batch. By minimizing the expected transport cost, it can determine the most likely label assignments. These two strategies work together to provide robust label selection from both local and global perspectives. Extensive experiments on eight benchmark datasets show that HopS consistently improves performance under partial supervision and outperforms all baselines. Those results highlight the merit of holistic label selection and offer a practical solution for prompt learning in weakly supervised settings.

Related papers

• Semi-Supervised Multi-Label Feature Selection with Consistent Sparse Graph Learning [13.401566810844368]
  Existing multi-label methods fail to evaluate the label correlations without enough labeled samples.<n>The similarity graph structure directly derived from the original feature space is suboptimal for multi-label problems.<n>We propose a consistent sparse graph learning method for multi-label semi-supervised feature selection.
  arXiv  Detail & Related papers  (2025-05-23T13:25:41Z)
• GPMFS: Global Foundation and Personalized Optimization for Multi-Label Feature Selection [0.820217860574125]
  The curse of dimensionality remains one of the major bottlenecks in high-dimensional multi-label learning.<n>We propose a novel method called GPMFS (Global Foundation and Personalized Optimization for Multi-Label Feature Selection)<n>Experiments on multiple real-world datasets demonstrate that GPMFS achieves superior performance while maintaining strong interpretability and robustness.
  arXiv  Detail & Related papers  (2025-04-17T08:29:14Z)
• Label Set Optimization via Activation Distribution Kurtosis for Zero-shot Classification with Generative Models [16.130133009174124]
  In-context learning (ICL) performance is highly sensitive to prompt design.<n>Class label options (e.g. lexicon or order) in zero-shot classification remains underexplored.<n>This study proposes LOADS, a method for selecting optimal label sets in zero-shot ICL with large language models.
  arXiv  Detail & Related papers  (2024-10-24T22:59:23Z)
• Pseudo-labelling meets Label Smoothing for Noisy Partial Label Learning [8.387189407144403]
  We motivate weakly supervised learning as an effective learning paradigm for problems where curating perfectly annotated datasets is expensive.<n>We focus on Partial Learning (PLL), a weakly-supervised learning paradigm where each training instance is paired with a set of candidate labels.<n>We present a framework that initially assigns pseudo-labels to images by exploiting the noisy partial labels through a weighted nearest neighbour algorithm.
  arXiv  Detail & Related papers  (2024-02-07T13:32:47Z)
• Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations [91.67511167969934]
  imprecise label learning (ILL) is a framework for the unification of learning with various imprecise label configurations. We demonstrate that ILL can seamlessly adapt to partial label learning, semi-supervised learning, noisy label learning, and, more importantly, a mixture of these settings.
  arXiv  Detail & Related papers  (2023-05-22T04:50:28Z)
• Ambiguity-Resistant Semi-Supervised Learning for Dense Object Detection [98.66771688028426]
  We propose a Ambiguity-Resistant Semi-supervised Learning (ARSL) for one-stage detectors. Joint-Confidence Estimation (JCE) is proposed to quantifies the classification and localization quality of pseudo labels. ARSL effectively mitigates the ambiguities and achieves state-of-the-art SSOD performance on MS COCO and PASCAL VOC.
  arXiv  Detail & Related papers  (2023-03-27T07:46:58Z)
• Label Matching Semi-Supervised Object Detection [85.99282969977541]
  Semi-supervised object detection has made significant progress with the development of mean teacher driven self-training. Label mismatch problem is not yet fully explored in the previous works, leading to severe confirmation bias during self-training. We propose a simple yet effective LabelMatch framework from two different yet complementary perspectives.
  arXiv  Detail & Related papers  (2022-06-14T05:59:41Z)
• Coping with Label Shift via Distributionally Robust Optimisation [72.80971421083937]
  We propose a model that minimises an objective based on distributionally robust optimisation (DRO) We then design and analyse a gradient descent-proximal mirror ascent algorithm tailored for large-scale problems to optimise the proposed objective.
  arXiv  Detail & Related papers  (2020-10-23T08:33:04Z)
• SPL-MLL: Selecting Predictable Landmarks for Multi-Label Learning [87.27700889147144]
  We propose to select a small subset of labels as landmarks which are easy to predict according to input (predictable) and can well recover the other possible labels (representative) We employ the Alternating Direction Method (ADM) to solve our problem. Empirical studies on real-world datasets show that our method achieves superior classification performance over other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-16T11:07:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.