Importance of negative sampling in weak label learning

• URL: http://arxiv.org/abs/2309.13227v1
• Date: Sat, 23 Sep 2023 01:11:15 GMT
• Title: Importance of negative sampling in weak label learning
• Authors: Ankit Shah, Fuyu Tang, Zelin Ye, Rita Singh, Bhiksha Raj
• Abstract summary: Weak-label learning is a challenging task that requires learning from data "bags" containing positive and negative instances. We study several sampling strategies that can measure the usefulness of negative instances for weak-label learning and select them accordingly. Our work reveals that negative instances are not all equally irrelevant, and selecting them wisely can benefit weak-label learning.
• Score: 33.97406573051897
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Weak-label learning is a challenging task that requires learning from data "bags" containing positive and negative instances, but only the bag labels are known. The pool of negative instances is usually larger than positive instances, thus making selecting the most informative negative instance critical for performance. Such a selection strategy for negative instances from each bag is an open problem that has not been well studied for weak-label learning. In this paper, we study several sampling strategies that can measure the usefulness of negative instances for weak-label learning and select them accordingly. We test our method on CIFAR-10 and AudioSet datasets and show that it improves the weak-label classification performance and reduces the computational cost compared to random sampling methods. Our work reveals that negative instances are not all equally irrelevant, and selecting them wisely can benefit weak-label learning.

Related papers

• CLAF: Contrastive Learning with Augmented Features for Imbalanced Semi-Supervised Learning [40.5117833362268]
  Semi-supervised learning and contrastive learning have been progressively combined to achieve better performances in popular applications. One common manner is assigning pseudo-labels to unlabeled samples and selecting positive and negative samples from pseudo-labeled samples to apply contrastive learning. We propose Contrastive Learning with Augmented Features (CLAF) to alleviate the scarcity of minority class samples in contrastive learning.
  arXiv  Detail & Related papers  (2023-12-15T08:27:52Z)
• Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination [62.18768931714238]
  We propose a novel False Negative Elimination (FNE) strategy to select negatives via sampling. The results demonstrate the superiority of our proposed false negative elimination strategy.
  arXiv  Detail & Related papers  (2023-08-08T16:31:43Z)
• Robust Positive-Unlabeled Learning via Noise Negative Sample Self-correction [48.929877651182885]
  Learning from positive and unlabeled data is known as positive-unlabeled (PU) learning in literature. We propose a new robust PU learning method with a training strategy motivated by the nature of human learning.
  arXiv  Detail & Related papers  (2023-08-01T04:34:52Z)
• Better Sampling of Negatives for Distantly Supervised Named Entity Recognition [39.264878763160766]
  We propose a simple and straightforward approach for selecting the top negative samples that have high similarities with all the positive samples for training. Our method achieves consistent performance improvements on four distantly supervised NER datasets.
  arXiv  Detail & Related papers  (2023-05-22T15:35:39Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Adaptive Positive-Unlabelled Learning via Markov Diffusion [0.0]
  Positive-Unlabelled (PU) learning is the machine learning setting in which only a set of positive instances are labelled. The principal aim of the algorithm is to identify a set of instances which are likely to contain positive instances that were originally unlabelled.
  arXiv  Detail & Related papers  (2021-08-13T10:25:47Z)
• Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces [64.23172847182109]
  We show that different negative sampling schemes implicitly trade-off performance on dominant versus rare labels. We provide a unified means to explicitly tackle both sampling bias, arising from working with a subset of all labels, and labeling bias, which is inherent to the data due to label imbalance.
  arXiv  Detail & Related papers  (2021-05-12T15:40:13Z)
• Are Fewer Labels Possible for Few-shot Learning? [81.89996465197392]
  Few-shot learning is challenging due to its very limited data and labels. Recent studies in big transfer (BiT) show that few-shot learning can greatly benefit from pretraining on large scale labeled dataset in a different domain. We propose eigen-finetuning to enable fewer shot learning by leveraging the co-evolution of clustering and eigen-samples in the finetuning.
  arXiv  Detail & Related papers  (2020-12-10T18:59:29Z)
• Contrastive Learning with Hard Negative Samples [80.12117639845678]
  We develop a new family of unsupervised sampling methods for selecting hard negative samples. A limiting case of this sampling results in a representation that tightly clusters each class, and pushes different classes as far apart as possible. The proposed method improves downstream performance across multiple modalities, requires only few additional lines of code to implement, and introduces no computational overhead.
  arXiv  Detail & Related papers  (2020-10-09T14:18:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.