Label Shift Quantification with Robustness Guarantees via Distribution Feature Matching

• URL: http://arxiv.org/abs/2306.04376v2
• Date: Sun, 2 Jul 2023 19:43:53 GMT
• Title: Label Shift Quantification with Robustness Guarantees via Distribution Feature Matching
• Authors: Bastien Dussap, Gilles Blanchard, Badr-Eddine Ch\'erief-Abdellatif
• Abstract summary: We first present a unifying framework, distribution feature matching (DFM), that recovers as particular instances various estimators introduced in previous literature. We then extend this analysis to study robustness of DFM procedures in the misspecified setting under departure from the exact label shift hypothesis. These theoretical findings are confirmed by a detailed numerical study on simulated and real-world datasets.
• Score: 3.2013172123155615
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Quantification learning deals with the task of estimating the target label distribution under label shift. In this paper, we first present a unifying framework, distribution feature matching (DFM), that recovers as particular instances various estimators introduced in previous literature. We derive a general performance bound for DFM procedures, improving in several key aspects upon previous bounds derived in particular cases. We then extend this analysis to study robustness of DFM procedures in the misspecified setting under departure from the exact label shift hypothesis, in particular in the case of contamination of the target by an unknown distribution. These theoretical findings are confirmed by a detailed numerical study on simulated and real-world datasets. We also introduce an efficient, scalable and robust version of kernel-based DFM using the Random Fourier Feature principle.

Related papers

• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional general score-mismatched diffusion samplers. We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions. This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Beyond Discrepancy: A Closer Look at the Theory of Distribution Shift [27.99789694038377]
  This work takes a closer look at the theory of distribution shift for a classifier from a source to a target distribution. We show when only unlabeled data from the target is sufficient, and when labeled target data is needed. In all cases, we provide rigorous theoretical guarantees in the large sample regime.
  arXiv  Detail & Related papers  (2024-05-29T15:00:19Z)
• Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation [49.827306773992376]
  Continual Test-Time Adaptation (CTTA) is proposed to migrate a source pre-trained model to continually changing target distributions. Our proposed method attains state-of-the-art performance in both classification and segmentation CTTA tasks.
  arXiv  Detail & Related papers  (2023-12-19T15:34:52Z)
• MMD-based Variable Importance for Distributional Random Forest [5.0459880125089]
  We introduce a variable importance algorithm for Distributional Random Forests (DRFs) We show that the introduced importance measure is consistent, exhibits high empirical performance on both real and simulated data, and outperforms competitors.
  arXiv  Detail & Related papers  (2023-10-18T17:12:29Z)
• Dr. FERMI: A Stochastic Distributionally Robust Fair Empirical Risk Minimization Framework [12.734559823650887]
  In the presence of distribution shifts, fair machine learning models may behave unfairly on test data. Existing algorithms require full access to data and cannot be used when small batches are used. This paper proposes the first distributionally robust fairness framework with convergence guarantees that do not require knowledge of the causal graph.
  arXiv  Detail & Related papers  (2023-09-20T23:25:28Z)
• Partial sequence labeling with structured Gaussian Processes [8.239028141030621]
  We propose structured Gaussian Processes for partial sequence labeling. It encodes uncertainty in the prediction and does not need extra effort for model selection and hyper parameter learning. It is evaluated on several sequence labeling tasks and the experimental results show the effectiveness of the proposed model.
  arXiv  Detail & Related papers  (2022-09-20T00:56:49Z)
• Robust Calibration with Multi-domain Temperature Scaling [86.07299013396059]
  We develop a systematic calibration model to handle distribution shifts by leveraging data from multiple domains. Our proposed method -- multi-domain temperature scaling -- uses the robustness in the domains to improve calibration under distribution shift.
  arXiv  Detail & Related papers  (2022-06-06T17:32:12Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Instrumental Variable-Driven Domain Generalization with Unobserved Confounders [53.735614014067394]
  Domain generalization (DG) aims to learn from multiple source domains a model that can generalize well on unseen target domains. We propose an instrumental variable-driven DG method (IV-DG) by removing the bias of the unobserved confounders with two-stage learning. In the first stage, it learns the conditional distribution of the input features of one domain given input features of another domain. In the second stage, it estimates the relationship by predicting labels with the learned conditional distribution.
  arXiv  Detail & Related papers  (2021-10-04T13:32:57Z)
• A Framework for Supervised Heterogeneous Transfer Learning using Dynamic Distribution Adaptation and Manifold Regularization [3.476077954140922]
  We present a framework called TLF that builds a classifier for the target domain having only few labeled training records. We handle distribution divergence by simultaneously optimizing the structural risk functional, joint distributions between domains, and the manifold consistency underlying marginal distributions. We evaluate TLF on seven publicly available natural datasets and compare the performance of TLF against the performance of eleven state-of-the-art techniques.
  arXiv  Detail & Related papers  (2021-08-27T14:00:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.