Class Probability Matching Using Kernel Methods for Label Shift Adaptation

• URL: http://arxiv.org/abs/2312.07282v1
• Date: Tue, 12 Dec 2023 13:59:37 GMT
• Title: Class Probability Matching Using Kernel Methods for Label Shift Adaptation
• Authors: Hongwei Wen, Annika Betken, Hanyuan Hang
• Abstract summary: We propose a new framework called textitclass probability matching (textitCPM) for label shift adaptation. By incorporating the kernel logistic regression into the CPM framework to estimate the conditional probability, we propose an algorithm called textitCPMKM for label shift adaptation.
• Score: 10.926835355554553
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In domain adaptation, covariate shift and label shift problems are two distinct and complementary tasks. In covariate shift adaptation where the differences in data distribution arise from variations in feature probabilities, existing approaches naturally address this problem based on \textit{feature probability matching} (\textit{FPM}). However, for label shift adaptation where the differences in data distribution stem solely from variations in class probability, current methods still use FPM on the $d$-dimensional feature space to estimate the class probability ratio on the one-dimensional label space. To address label shift adaptation more naturally and effectively, inspired by a new representation of the source domain's class probability, we propose a new framework called \textit{class probability matching} (\textit{CPM}) which matches two class probability functions on the one-dimensional label space to estimate the class probability ratio, fundamentally different from FPM operating on the $d$-dimensional feature space. Furthermore, by incorporating the kernel logistic regression into the CPM framework to estimate the conditional probability, we propose an algorithm called \textit{class probability matching using kernel methods} (\textit{CPMKM}) for label shift adaptation. From the theoretical perspective, we establish the optimal convergence rates of CPMKM with respect to the cross-entropy loss for multi-class label shift adaptation. From the experimental perspective, comparisons on real datasets demonstrate that CPMKM outperforms existing FPM-based and maximum-likelihood-based algorithms.

Related papers

• Improving Distribution Alignment with Diversity-based Sampling [0.0]
  Domain shifts are ubiquitous in machine learning, and can substantially degrade a model's performance when deployed to real-world data. This paper proposes to improve these estimates by inducing diversity in each sampled minibatch. It simultaneously balances the data and reduces the variance of the gradients, thereby enhancing the model's generalisation ability.
  arXiv  Detail & Related papers  (2024-10-05T17:26:03Z)
• Variational Classification [51.2541371924591]
  We derive a variational objective to train the model, analogous to the evidence lower bound (ELBO) used to train variational auto-encoders. Treating inputs to the softmax layer as samples of a latent variable, our abstracted perspective reveals a potential inconsistency. We induce a chosen latent distribution, instead of the implicit assumption found in a standard softmax layer.
  arXiv  Detail & Related papers  (2023-05-17T17:47:19Z)
• RLSbench: Domain Adaptation Under Relaxed Label Shift [39.845383643588356]
  We introduce RLSbench, a large-scale benchmark for relaxed label shift. We assess 13 popular domain adaptation methods, demonstrating more widespread failures under label proportion shifts than were previously known. We develop an effective two-step meta-algorithm that is compatible with most domain adaptations.
  arXiv  Detail & Related papers  (2023-02-06T18:57:14Z)
• Cycle Label-Consistent Networks for Unsupervised Domain Adaptation [57.29464116557734]
  Domain adaptation aims to leverage a labeled source domain to learn a classifier for the unlabeled target domain with a different distribution. We propose a simple yet efficient domain adaptation method, i.e. Cycle Label-Consistent Network (CLCN), by exploiting the cycle consistency of classification label. We demonstrate the effectiveness of our approach on MNIST-USPS-SVHN, Office-31, Office-Home and Image CLEF-DA benchmarks.
  arXiv  Detail & Related papers  (2022-05-27T13:09:08Z)
• PPFS: Predictive Permutation Feature Selection [2.502407331311937]
  We propose a novel wrapper-based feature selection method based on the concept of Markov Blanket (MB) Unlike previous MB methods, PPFS is a universal feature selection technique as it can work for both classification and regression tasks. We propose Predictive Permutation Independence (PPI), a new Conditional Independence (CI) test, which enables PPFS to be categorised as a wrapper feature selection method.
  arXiv  Detail & Related papers  (2021-10-20T18:18:18Z)
• PLM: Partial Label Masking for Imbalanced Multi-label Classification [59.68444804243782]
  Neural networks trained on real-world datasets with long-tailed label distributions are biased towards frequent classes and perform poorly on infrequent classes. We propose a method, Partial Label Masking (PLM), which utilizes this ratio during training. Our method achieves strong performance when compared to existing methods on both multi-label (MultiMNIST and MSCOCO) and single-label (imbalanced CIFAR-10 and CIFAR-100) image classification datasets.
  arXiv  Detail & Related papers  (2021-05-22T18:07:56Z)
• On Universal Black-Box Domain Adaptation [53.7611757926922]
  We study an arguably least restrictive setting of domain adaptation in a sense of practical deployment. Only the interface of source model is available to the target domain, and where the label-space relations between the two domains are allowed to be different and unknown. We propose to unify them into a self-training framework, regularized by consistency of predictions in local neighborhoods of target samples.
  arXiv  Detail & Related papers  (2021-04-10T02:21:09Z)
• A Unified Joint Maximum Mean Discrepancy for Domain Adaptation [73.44809425486767]
  This paper theoretically derives a unified form of JMMD that is easy to optimize. From the revealed unified JMMD, we illustrate that JMMD degrades the feature-label dependence that benefits to classification. We propose a novel MMD matrix to promote the dependence, and devise a novel label kernel that is robust to label distribution shift.
  arXiv  Detail & Related papers  (2021-01-25T09:46:14Z)
• Coping with Label Shift via Distributionally Robust Optimisation [72.80971421083937]
  We propose a model that minimises an objective based on distributionally robust optimisation (DRO) We then design and analyse a gradient descent-proximal mirror ascent algorithm tailored for large-scale problems to optimise the proposed objective.
  arXiv  Detail & Related papers  (2020-10-23T08:33:04Z)
• Posterior Re-calibration for Imbalanced Datasets [33.379680556475314]
  Neural Networks can perform poorly when the training label distribution is heavily imbalanced. We derive a post-training prior rebalancing technique that can be solved through a KL-divergence based optimization. Our results on six different datasets and five different architectures show state of art accuracy.
  arXiv  Detail & Related papers  (2020-10-22T15:57:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.