Verifying the Selected Completely at Random Assumption in Positive-Unlabeled Learning

• URL: http://arxiv.org/abs/2404.00145v1
• Date: Fri, 29 Mar 2024 20:36:58 GMT
• Title: Verifying the Selected Completely at Random Assumption in Positive-Unlabeled Learning
• Authors: Paweł Teisseyre, Konrad Furmańczyk, Jan Mielniczuk,
• Abstract summary: We propose a relatively simple and computationally fast test that can be used to determine whether the observed data meet the SCAR assumption. Our test is based on generating artificial labels conforming to the SCAR case, which in turn allows to mimic the distribution of the test statistic under the null hypothesis of SCAR.
• Score: 0.7646713951724013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The goal of positive-unlabeled (PU) learning is to train a binary classifier on the basis of training data containing positive and unlabeled instances, where unlabeled observations can belong either to the positive class or to the negative class. Modeling PU data requires certain assumptions on the labeling mechanism that describes which positive observations are assigned a label. The simplest assumption, considered in early works, is SCAR (Selected Completely at Random Assumption), according to which the propensity score function, defined as the probability of assigning a label to a positive observation, is constant. On the other hand, a much more realistic assumption is SAR (Selected at Random), which states that the propensity function solely depends on the observed feature vector. SCAR-based algorithms are much simpler and computationally much faster compared to SAR-based algorithms, which usually require challenging estimation of the propensity score. In this work, we propose a relatively simple and computationally fast test that can be used to determine whether the observed data meet the SCAR assumption. Our test is based on generating artificial labels conforming to the SCAR case, which in turn allows to mimic the distribution of the test statistic under the null hypothesis of SCAR. We justify our method theoretically. In experiments, we demonstrate that the test successfully detects various deviations from SCAR scenario and at the same time it is possible to effectively control the type I error. The proposed test can be recommended as a pre-processing step to decide which final PU algorithm to choose in cases when nature of labeling mechanism is not known.

Related papers

• Augmented prediction of a true class for Positive Unlabeled data under selection bias [0.8594140167290099]
  We introduce a new observational setting for Positive Unlabeled (PU) data where the observations at prediction time are also labeled. We argue that the additional information is important for prediction, and call this task "augmented PU prediction" We introduce several variants of the empirical Bayes rule in such scenario and investigate their performance.
  arXiv  Detail & Related papers  (2024-07-14T19:58:01Z)
• Joint empirical risk minimization for instance-dependent positive-unlabeled data [4.112909937203119]
  Learning from positive and unlabeled data (PU learning) is actively researched machine learning task. The goal is to train a binary classification model based on a dataset containing part on positives which are labeled, and unlabeled instances. Unlabeled set includes remaining part positives and all negative observations.
  arXiv  Detail & Related papers  (2023-12-27T12:45:12Z)
• Positive Unlabeled Learning Selected Not At Random (PULSNAR): class proportion estimation when the SCAR assumption does not hold [2.76815720120527]
  Positive and Unlabeled (PU) learning is a type of semi-supervised binary classification. PU learning has broad applications in settings where confirmed negatives are unavailable or difficult to obtain. We propose two PU learning algorithms to estimate $alpha$, calculate probabilities for PU instances, and improve classification metrics.
  arXiv  Detail & Related papers  (2023-03-14T23:16:22Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Risk bounds for PU learning under Selected At Random assumption [0.0]
  Positive-unlabeled learning (PU learning) is known as a special case of semi-supervised binary classification where only a fraction of positive examples are labeled. We provide a lower bound on minimax risk proving that the upper bound is almost optimal.
  arXiv  Detail & Related papers  (2022-01-17T08:45:39Z)
• Learning with Proper Partial Labels [87.65718705642819]
  Partial-label learning is a kind of weakly-supervised learning with inexact labels. We show that this proper partial-label learning framework includes many previous partial-label learning settings. We then derive a unified unbiased estimator of the classification risk.
  arXiv  Detail & Related papers  (2021-12-23T01:37:03Z)
• Positive-Unlabeled Classification under Class-Prior Shift: A Prior-invariant Approach Based on Density Ratio Estimation [85.75352990739154]
  We propose a novel PU classification method based on density ratio estimation. A notable advantage of our proposed method is that it does not require the class-priors in the training phase.
  arXiv  Detail & Related papers  (2021-07-11T13:36:53Z)
• A Statistical Test for Probabilistic Fairness [11.95891442664266]
  We propose a statistical hypothesis test for detecting unfair classifiers. We show both theoretically as well as empirically that the proposed test is correct. In addition, the proposed framework offers interpretability by identifying the most favorable perturbation of the data.
  arXiv  Detail & Related papers  (2020-12-09T00:20:02Z)
• Semi-Supervised Speech Recognition via Graph-based Temporal Classification [59.58318952000571]
  Semi-supervised learning has demonstrated promising results in automatic speech recognition by self-training. The effectiveness of this approach largely relies on the pseudo-label accuracy. Alternative ASR hypotheses of an N-best list can provide more accurate labels for an unlabeled speech utterance.
  arXiv  Detail & Related papers  (2020-10-29T14:56:56Z)
• Learning with Out-of-Distribution Data for Audio Classification [60.48251022280506]
  We show that detecting and relabelling certain OOD instances, rather than discarding them, can have a positive effect on learning. The proposed method is shown to improve the performance of convolutional neural networks by a significant margin.
  arXiv  Detail & Related papers  (2020-02-11T21:08:06Z)
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing [105.91827623768724]
  Machine learning algorithms are susceptible to data poisoning attacks. We present a unifying view of randomized smoothing over arbitrary functions. We propose a new strategy for building classifiers that are pointwise-certifiably robust to general data poisoning attacks.
  arXiv  Detail & Related papers  (2020-02-07T21:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.