Theory-inspired Label Shift Adaptation via Aligned Distribution Mixture

• URL: http://arxiv.org/abs/2411.02047v2
• Date: Tue, 05 Nov 2024 09:09:29 GMT
• Title: Theory-inspired Label Shift Adaptation via Aligned Distribution Mixture
• Authors: Ruidong Fan, Xiao Ouyang, Hong Tao, Yuhua Qian, Chenping Hou,
• Abstract summary: We propose an innovative label shift framework named as Aligned Distribution Mixture (ADM) Within this framework, we enhance four typical label shift methods by introducing modifications to the classifier training process. Considering the distinctiveness of the proposed one-step approach, we develop an efficient bi-level optimization strategy.
• Score: 21.494268411607766
• License:
• Abstract: As a prominent challenge in addressing real-world issues within a dynamic environment, label shift, which refers to the learning setting where the source (training) and target (testing) label distributions do not match, has recently received increasing attention. Existing label shift methods solely use unlabeled target samples to estimate the target label distribution, and do not involve them during the classifier training, resulting in suboptimal utilization of available information. One common solution is to directly blend the source and target distributions during the training of the target classifier. However, we illustrate the theoretical deviation and limitations of the direct distribution mixture in the label shift setting. To tackle this crucial yet unexplored issue, we introduce the concept of aligned distribution mixture, showcasing its theoretical optimality and generalization error bounds. By incorporating insights from generalization theory, we propose an innovative label shift framework named as Aligned Distribution Mixture (ADM). Within this framework, we enhance four typical label shift methods by introducing modifications to the classifier training process. Furthermore, we also propose a one-step approach that incorporates a pioneering coupling weight estimation strategy. Considering the distinctiveness of the proposed one-step approach, we develop an efficient bi-level optimization strategy. Experimental results demonstrate the effectiveness of our approaches, together with their effectiveness in COVID-19 diagnosis applications.

Related papers

• Harnessing Hierarchical Label Distribution Variations in Test Agnostic Long-tail Recognition [114.96385572118042]
  We argue that the variation in test label distributions can be broken down hierarchically into global and local levels. We propose a new MoE strategy, $mathsfDirMixE$, which assigns experts to different Dirichlet meta-distributions of the label distribution. We show that our proposed objective benefits from enhanced generalization by virtue of the variance-based regularization.
  arXiv  Detail & Related papers  (2024-05-13T14:24:56Z)
• Distribution Consistency based Self-Training for Graph Neural Networks with Sparse Labels [33.89511660654271]
  Few-shot node classification poses a significant challenge for Graph Neural Networks (GNNs) Self-training has emerged as a widely popular framework to leverage the abundance of unlabeled data. We propose a novel Distribution-Consistent Graph Self-Training framework to identify pseudo-labeled nodes that are both informative and capable of redeeming the distribution discrepancy.
  arXiv  Detail & Related papers  (2024-01-18T22:07:48Z)
• RankMatch: A Novel Approach to Semi-Supervised Label Distribution Learning Leveraging Inter-label Correlations [52.549807652527306]
  This paper introduces RankMatch, an innovative approach for Semi-Supervised Label Distribution Learning (SSLDL) RankMatch effectively utilizes a small number of labeled examples in conjunction with a larger quantity of unlabeled data. We establish a theoretical generalization bound for RankMatch, and through extensive experiments, demonstrate its superiority in performance against existing SSLDL methods.
  arXiv  Detail & Related papers  (2023-12-11T12:47:29Z)
• GeT: Generative Target Structure Debiasing for Domain Adaptation [67.17025068995835]
  Domain adaptation (DA) aims to transfer knowledge from a fully labeled source to a scarcely labeled or totally unlabeled target under domain shift. Recently, semi-supervised learning-based (SSL) techniques that leverage pseudo labeling have been increasingly used in DA. In this paper, we propose GeT that learns a non-bias target embedding distribution with high quality pseudo labels.
  arXiv  Detail & Related papers  (2023-08-20T08:52:43Z)
• Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label Learning [97.88458953075205]
  Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data. This paper proposes a novel solution called Class-Aware Pseudo-Labeling (CAP) that performs pseudo-labeling in a class-aware manner.
  arXiv  Detail & Related papers  (2023-05-04T12:52:18Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Combating Label Distribution Shift for Active Domain Adaptation [16.270897459117755]
  We consider the problem of active domain adaptation (ADA) to unlabeled target data. Inspired by recent analysis on a critical issue from label distribution mismatch between source and target in domain adaptation, we devise a method that addresses the issue for the first time in ADA.
  arXiv  Detail & Related papers  (2022-08-13T09:06:45Z)
• Domain Adaptation under Open Set Label Shift [39.424134505152544]
  We introduce the problem of domain adaptation under Open Set Label Shift (OSLS) OSLS subsumes domain adaptation under label shift and Positive-Unlabeled (PU) learning. We propose practical methods for both tasks that leverage black-box predictors.
  arXiv  Detail & Related papers  (2022-07-26T17:09:48Z)
• Similarity Based Label Smoothing For Dialogue Generation [1.1279808969568252]
  Generative neural conversational systems are generally trained with the objective of minimizing the entropy loss between the training "hard" targets and the predicted logits. Label smoothing enforces a data independent uniform distribution on the incorrect training targets, which leads to an incorrect assumption of equi-probable incorrect targets for each correct target. We propose to transform the uniform distribution of the incorrect target probabilities in label smoothing, to a more natural distribution based on semantics.
  arXiv  Detail & Related papers  (2021-07-23T23:25:19Z)
• MatchGAN: A Self-Supervised Semi-Supervised Conditional Generative Adversarial Network [51.84251358009803]
  We present a novel self-supervised learning approach for conditional generative adversarial networks (GANs) under a semi-supervised setting. We perform augmentation by randomly sampling sensible labels from the label space of the few labelled examples available. Our method surpasses the baseline with only 20% of the labelled examples used to train the baseline.
  arXiv  Detail & Related papers  (2020-06-11T17:14:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.