Easy Learning from Label Proportions

• URL: http://arxiv.org/abs/2302.03115v1
• Date: Mon, 6 Feb 2023 20:41:38 GMT
• Title: Easy Learning from Label Proportions
• Authors: Robert Istvan Busa-Fekete, Heejin Choi, Travis Dick, Claudio Gentile, Andres Munoz medina
• Abstract summary: Easyllp is a flexible and simple-to-implement debiasing approach based on aggregate labels. Our technique allows us to accurately estimate the expected loss of an arbitrary model at an individual level.
• Score: 17.71834385754893
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of Learning from Label Proportions (LLP), a weakly supervised classification setup where instances are grouped into "bags", and only the frequency of class labels at each bag is available. Albeit, the objective of the learner is to achieve low task loss at an individual instance level. Here we propose Easyllp: a flexible and simple-to-implement debiasing approach based on aggregate labels, which operates on arbitrary loss functions. Our technique allows us to accurately estimate the expected loss of an arbitrary model at an individual level. We showcase the flexibility of our approach by applying it to popular learning frameworks, like Empirical Risk Minimization (ERM) and Stochastic Gradient Descent (SGD) with provable guarantees on instance level performance. More concretely, we exhibit a variance reduction technique that makes the quality of LLP learning deteriorate only by a factor of k (k being bag size) in both ERM and SGD setups, as compared to full supervision. Finally, we validate our theoretical results on multiple datasets demonstrating our algorithm performs as well or better than previous LLP approaches in spite of its simplicity.

Related papers

• Forming Auxiliary High-confident Instance-level Loss to Promote Learning from Label Proportions [17.36538357653019]
  Learning from label proportions (LLP) aims to train a classifier by using bags of instances and the proportions of classes within bags, rather than annotated labels for each instance. We propose a novel LLP method, namely Learning from Label Proportions with Auxiliary High-confident Instance-level Loss (L2P-AHIL) We show that L2P-AHIL can surpass the existing baseline methods, and the performance gain can be more significant as the bag size increases.
  arXiv  Detail & Related papers  (2024-11-15T17:14:18Z)
• Uncertainty Aware Learning for Language Model Alignment [97.36361196793929]
  We propose uncertainty-aware learning (UAL) to improve the model alignment of different task scenarios. We implement UAL in a simple fashion -- adaptively setting the label smoothing value of training according to the uncertainty of individual samples. Experiments on widely used benchmarks demonstrate that our UAL significantly and consistently outperforms standard supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-06-07T11:37:45Z)
• Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach [102.0769560460338]
  We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
  arXiv  Detail & Related papers  (2024-03-01T03:27:08Z)
• Simple and Fast Group Robustness by Automatic Feature Reweighting [45.9024045614187]
  We propose Automatic Feature Reweighting (AFR) to reduce the reliance on spurious features. AFR retrains the last layer of a standard ERM-trained base model with a weighted loss. We improve upon the best reported results among competing methods trained without spurious attributes on several vision and natural language classification benchmarks.
  arXiv  Detail & Related papers  (2023-06-19T17:19:13Z)
• Active Learning in the Predict-then-Optimize Framework: A Margin-Based Approach [5.371816551086118]
  We develop a learning method that sequentially decides whether to request the "labels" of feature samples from an unlabeled data stream. Our active learning method is the first to be directly informed by the decision error induced by the predicted parameters.
  arXiv  Detail & Related papers  (2023-05-11T05:44:36Z)
• SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning [101.86916775218403]
  This paper revisits the popular pseudo-labeling methods via a unified sample weighting formulation. We propose SoftMatch to overcome the trade-off by maintaining both high quantity and high quality of pseudo-labels during training. In experiments, SoftMatch shows substantial improvements across a wide variety of benchmarks, including image, text, and imbalanced classification.
  arXiv  Detail & Related papers  (2023-01-26T03:53:25Z)
• Learning from Label Proportions by Learning with Label Noise [30.7933303912474]
  Learning from label proportions (LLP) is a weakly supervised classification problem where data points are grouped into bags. We provide a theoretically grounded approach to LLP based on a reduction to learning with label noise. Our approach demonstrates improved empirical performance in deep learning scenarios across multiple datasets and architectures.
  arXiv  Detail & Related papers  (2022-03-04T18:52:21Z)
• Sum of Ranked Range Loss for Supervised Learning [47.0464265614452]
  We introduce the sum of ranked range (SoRR) as a general approach to form learning objectives. A ranked range is a consecutive sequence of sorted values of a set of real numbers. We explore two applications in machine learning of the minimization of the SoRR framework, namely the AoRR aggregate loss for binary/multi-class classification at the sample level and the TKML individual loss for multi-label/multi-class classification at the label level.
  arXiv  Detail & Related papers  (2021-06-07T02:11:27Z)
• Attentional-Biased Stochastic Gradient Descent [74.49926199036481]
  We present a provable method (named ABSGD) for addressing the data imbalance or label noise problem in deep learning. Our method is a simple modification to momentum SGD where we assign an individual importance weight to each sample in the mini-batch. ABSGD is flexible enough to combine with other robust losses without any additional cost.
  arXiv  Detail & Related papers  (2020-12-13T03:41:52Z)
• Learning by Minimizing the Sum of Ranked Range [58.24935359348289]
  We introduce the sum of ranked range (SoRR) as a general approach to form learning objectives. A ranked range is a consecutive sequence of sorted values of a set of real numbers. We explore two applications in machine learning of the minimization of the SoRR framework, namely the AoRR aggregate loss for binary classification and the TKML individual loss for multi-label/multi-class classification.
  arXiv  Detail & Related papers  (2020-10-05T01:58:32Z)
• Progressive Identification of True Labels for Partial-Label Learning [112.94467491335611]
  Partial-label learning (PLL) is a typical weakly supervised learning problem, where each training instance is equipped with a set of candidate labels among which only one is the true label. Most existing methods elaborately designed as constrained optimizations that must be solved in specific manners, making their computational complexity a bottleneck for scaling up to big data. This paper proposes a novel framework of classifier with flexibility on the model and optimization algorithm.
  arXiv  Detail & Related papers  (2020-02-19T08:35:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.