Learning from Label Proportions: Bootstrapping Supervised Learners via Belief Propagation

• URL: http://arxiv.org/abs/2310.08056v4
• Date: Wed, 20 Mar 2024 07:23:32 GMT
• Title: Learning from Label Proportions: Bootstrapping Supervised Learners via Belief Propagation
• Authors: Shreyas Havaldar, Navodita Sharma, Shubhi Sareen, Karthikeyan Shanmugam, Aravindan Raghuveer,
• Abstract summary: Learning from Label Proportions (LLP) is a learning problem where only aggregate level labels are available for groups of instances, called bags, during training. This setting arises in domains like advertising and medicine due to privacy considerations. We propose a novel algorithmic framework for this problem that iteratively performs two main steps.
• Score: 18.57840057487926
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning from Label Proportions (LLP) is a learning problem where only aggregate level labels are available for groups of instances, called bags, during training, and the aim is to get the best performance at the instance-level on the test data. This setting arises in domains like advertising and medicine due to privacy considerations. We propose a novel algorithmic framework for this problem that iteratively performs two main steps. For the first step (Pseudo Labeling) in every iteration, we define a Gibbs distribution over binary instance labels that incorporates a) covariate information through the constraint that instances with similar covariates should have similar labels and b) the bag level aggregated label. We then use Belief Propagation (BP) to marginalize the Gibbs distribution to obtain pseudo labels. In the second step (Embedding Refinement), we use the pseudo labels to provide supervision for a learner that yields a better embedding. Further, we iterate on the two steps again by using the second step's embeddings as new covariates for the next iteration. In the final iteration, a classifier is trained using the pseudo labels. Our algorithm displays strong gains against several SOTA baselines (up to 15%) for the LLP Binary Classification problem on various dataset types - tabular and Image. We achieve these improvements with minimal computational overhead above standard supervised learning due to Belief Propagation, for large bag sizes, even for a million samples.

Related papers

• Active Generalized Category Discovery [60.69060965936214]
  Generalized Category Discovery (GCD) endeavors to cluster unlabeled samples from both novel and old classes. We take the spirit of active learning and propose a new setting called Active Generalized Category Discovery (AGCD) Our method achieves state-of-the-art performance on both generic and fine-grained datasets.
  arXiv  Detail & Related papers  (2024-03-07T07:12:24Z)
• Pseudo-labelling meets Label Smoothing for Noisy Partial Label Learning [8.387189407144403]
  Partial label learning (PLL) is a weakly-supervised learning paradigm where each training instance is paired with a set of candidate labels (partial label) NPLL relaxes this constraint by allowing some partial labels to not contain the true label, enhancing the practicality of the problem. We present a minimalistic framework that initially assigns pseudo-labels to images by exploiting the noisy partial labels through a weighted nearest neighbour algorithm.
  arXiv  Detail & Related papers  (2024-02-07T13:32:47Z)
• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• Partial-Label Regression [54.74984751371617]
  Partial-label learning is a weakly supervised learning setting that allows each training example to be annotated with a set of candidate labels. Previous studies on partial-label learning only focused on the classification setting where candidate labels are all discrete. In this paper, we provide the first attempt to investigate partial-label regression, where each training example is annotated with a set of real-valued candidate labels.
  arXiv  Detail & Related papers  (2023-06-15T09:02:24Z)
• Learning from Stochastic Labels [8.178975818137937]
  Annotating multi-class instances is a crucial task in the field of machine learning. In this paper, we propose a novel suitable approach to learn from these labels.
  arXiv  Detail & Related papers  (2023-02-01T08:04:27Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Instance-Dependent Partial Label Learning [69.49681837908511]
  Partial label learning is a typical weakly supervised learning problem. Most existing approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels. In this paper, we consider instance-dependent and assume that each example is associated with a latent label distribution constituted by the real number of each label.
  arXiv  Detail & Related papers  (2021-10-25T12:50:26Z)
• Active Learning in Incomplete Label Multiple Instance Multiple Label Learning [17.5720245903743]
  We propose a novel bag-class pair based approach for active learning in the MIML setting. Our approach is based on a discriminative graphical model with efficient and exact inference.
  arXiv  Detail & Related papers  (2021-07-22T17:01:28Z)
• All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training [32.45488147013166]
  Pseudo-labeling is a key component in semi-supervised learning (SSL) We propose SemCo, a method which leverages label semantics and co-training to address this problem. We show that our method achieves state-of-the-art performance across various SSL tasks including 5.6% accuracy improvement on Mini-ImageNet dataset with 1000 labeled examples.
  arXiv  Detail & Related papers  (2021-04-12T07:33:16Z)
• A Study on the Autoregressive and non-Autoregressive Multi-label Learning [77.11075863067131]
  We propose a self-attention based variational encoder-model to extract the label-label and label-feature dependencies jointly. Our model can therefore be used to predict all labels in parallel while still including both label-label and label-feature dependencies.
  arXiv  Detail & Related papers  (2020-12-03T05:41:44Z)
• Instance Credibility Inference for Few-Shot Learning [45.577880041135785]
  Few-shot learning aims to recognize new objects with extremely limited training data for each category. This paper presents a simple statistical approach, dubbed Instance Credibility Inference (ICI) to exploit the distribution support of unlabeled instances for few-shot learning. Our simple approach can establish new state-of-the-arts on four widely used few-shot learning benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-26T12:01:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.