In-Context Positive-Unlabeled Learning

• URL: http://arxiv.org/abs/2605.05591v1
• Date: Thu, 07 May 2026 02:17:53 GMT
• Title: In-Context Positive-Unlabeled Learning
• Abstract summary: We introduce PUICL, a pretrained transformer that solves PU classification entirely through in-supervised learning.<n> PUICL is pretrained on synthetic PU datasets generated from randomly instantiated structural causal models.<n>On 20 semi-synthetic PU benchmarks, PUICL outperforms four standard PU learning baselines in average AUC and accuracy, and is competitive on F1-score.
• Score: 15.73766898767159
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Positive-unlabeled (PU) learning addresses binary classification when only a set of labeled positives is available alongside a pool of unlabeled samples drawn from a mixture of positives and negatives. Existing PU methods typically require dataset-specific training or iterative optimization, which limits their applicability when many tasks must be solved quickly or with little tuning. We introduce PUICL, a pretrained transformer that solves PU classification entirely through in-context learning. PUICL is pretrained on synthetic PU datasets generated from randomly instantiated structural causal models, exposing it to a wide range of feature-label relationships and class-prior configurations. At inference time, PUICL receives the labeled positives and the unlabeled samples as a single input and returns class probabilities for the unlabeled rows in one forward pass, with no gradient updates or per-task fitting. On 20 semi-synthetic PU benchmarks derived from the UCI Machine Learning Repository, OpenML, and scikit-learn, PUICL outperforms four standard PU learning baselines in average AUC and accuracy, and is competitive on F1-score. These results show that the in-context learning paradigm extends naturally beyond fully supervised tabular prediction to the semi-supervised PU setting.

Related papers

• Noisy-Pair Robust Representation Alignment for Positive-Unlabeled Learning [24.345089357698985]
  Negative-Unlabeled (PU) learning aims to train a binary classifier where only limited positive data and abundant unlabeled data are available.<n>We propose NcPU, a non-contrastive PU learning framework that requires no auxiliary information.<n>We show that NcPU achieves substantial improvements over state-of-the-art PU methods across diverse datasets.
  arXiv  Detail & Related papers  (2025-09-30T18:22:30Z)
• Accessible, Realistic, and Fair Evaluation of Positive-Unlabeled Learning Algorithms [54.58593451541316]
  We propose the first PU learning benchmark to systematically compare PU learning algorithms.<n>We identify subtle yet critical factors that affect the realistic and fair evaluation of PU learning algorithms.
  arXiv  Detail & Related papers  (2025-09-29T03:13:00Z)
• Probably Approximately Precision and Recall Learning [60.00180898830079]
  A key challenge in machine learning is the prevalence of one-sided feedback.<n>We introduce a Probably Approximately Correct (PAC) framework in which hypotheses are set functions that map each input to a set of labels.<n>We develop new algorithms that learn from positive data alone, achieving optimal sample complexity in the realizable case.
  arXiv  Detail & Related papers  (2024-11-20T04:21:07Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• Understanding Contrastive Representation Learning from Positive Unlabeled (PU) Data [28.74519165747641]
  We study the problem of Positive Unlabeled (PU) learning, where only a small set of labeled positives and a large unlabeled pool are available.<n>We introduce Positive Unlabeled Contrastive Learning (puCL), an unbiased and variance reducing contrastive objective.<n>When the class prior is known, we propose Positive Unlabeled InfoNCE (puNCE), a prior-aware extension that re-weights unlabeled samples as soft positive negative mixtures.
  arXiv  Detail & Related papers  (2024-02-08T20:20:54Z)
• Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical [66.57396042747706]
  Complementary-label learning is a weakly supervised learning problem. We propose a consistent approach that does not rely on the uniform distribution assumption. We find that complementary-label learning can be expressed as a set of negative-unlabeled binary classification problems.
  arXiv  Detail & Related papers  (2023-11-27T02:59:17Z)
• CLIPood: Generalizing CLIP to Out-of-Distributions [73.86353105017076]
  Contrastive language-image pre-training (CLIP) models have shown impressive zero-shot ability, but the further adaptation of CLIP on downstream tasks undesirably degrades OOD performances. We propose CLIPood, a fine-tuning method that can adapt CLIP models to OOD situations where both domain shifts and open classes may occur on unseen test data. Experiments on diverse datasets with different OOD scenarios show that CLIPood consistently outperforms existing generalization techniques.
  arXiv  Detail & Related papers  (2023-02-02T04:27:54Z)
• Positive Unlabeled Contrastive Learning [14.975173394072053]
  We extend the self-supervised pretraining paradigm to the classical positive unlabeled (PU) setting. We develop a simple methodology to pseudo-label the unlabeled samples using a new PU-specific clustering scheme. Our method handily outperforms state-of-the-art PU methods over several standard PU benchmark datasets.
  arXiv  Detail & Related papers  (2022-06-01T20:16:32Z)
• Uncertainty-aware Pseudo-label Selection for Positive-Unlabeled Learning [10.014356492742074]
  We propose to tackle the issues of imbalanced datasets and model calibration in a positive-unlabeled learning setting. By boosting the signal from the minority class, pseudo-labeling expands the labeled dataset with new samples from the unlabeled set. Within a series of experiments, PUUPL yields substantial performance gains in highly imbalanced settings.
  arXiv  Detail & Related papers  (2022-01-31T12:55:47Z)
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning [53.1047775185362]
  Pseudo-labeling (PL) is a general SSL approach that does not have this constraint but performs relatively poorly in its original formulation. We argue that PL underperforms due to the erroneous high confidence predictions from poorly calibrated models. We propose an uncertainty-aware pseudo-label selection (UPS) framework which improves pseudo labeling accuracy by drastically reducing the amount of noise encountered in the training process.
  arXiv  Detail & Related papers  (2021-01-15T23:29:57Z)
• On Leveraging Unlabeled Data for Concurrent Positive-Unlabeled Classification and Robust Generation [72.062661402124]
  We present a novel training framework to jointly target PU classification and conditional generation when exposed to extra data.<n>We prove the optimal condition of CNI-CGAN and experimentally, we conducted extensive evaluations on diverse datasets.
  arXiv  Detail & Related papers  (2020-06-14T08:27:40Z)
• Improving Positive Unlabeled Learning: Practical AUL Estimation and New Training Method for Extremely Imbalanced Data Sets [10.870831090350402]
  We improve Positive Unlabeled (PU) learning over state-of-the-art from two aspects. First, we propose an unbiased practical AUL estimation method, which makes use of raw PU data without prior knowledge of unlabeled samples. Secondly, we propose ProbTagging, a new training method for extremely imbalanced data sets.
  arXiv  Detail & Related papers  (2020-04-21T08:32:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.