Informative Label Missingness in Multiclass Classification Information Geometry and Excess Risk

• URL: http://arxiv.org/abs/2608.30561v1
• Date: Mon, 31 Aug 2026 10:37:49 GMT
• Title: Informative Label Missingness in Multiclass Classification Information Geometry and Excess Risk
• Abstract summary: We develop a general likelihood-based theory for the phenomenon in parametric multiclass classification.<n>An efficient-information decomposition separates information lost through unavailable class memberships from information contributed by the missing-label mechanism.<n>Near missing completely at random, with the marginal missing-label proportion fixed, redistribution of missing labels changes lost class-label information at first order, whereas efficient information from the missingness pattern appears only at second order.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Informative label missingness can change the usual efficiency ordering between completely and partially labelled classifiers because the pattern of missing labels may itself carry information about the classification model. We develop a general likelihood-based theory for this phenomenon in parametric multiclass classification. An efficient-information decomposition separates information lost through unavailable class memberships from information contributed by the missing-label mechanism. We then derive a quadratic expansion of plug-in excess risk over the active pairwise faces of the multiclass Bayes boundary, showing that classification efficiency depends on how information gains and losses align with directions that perturb the decision boundary. This yields a classification-weighted generalized-eigenvalue criterion under which informative partial classification may have smaller asymptotic classification risk without globally dominating complete classification in Fisher information. Near missing completely at random, with the marginal missing-label proportion fixed, redistribution of missing labels changes lost class-label information at first order, whereas efficient information from the missingness pattern appears only at second order. Three-class quadratic discriminant calculations, finite-sample experiments, and a semi-synthetic multiclass application illustrate the resulting regime-dependent behaviour.

Related papers

• Semi-Supervised Classification with Informative Missing Labels in Weibull Mixture Models [2.5794915063815664]
  We consider semi-supervised classification from a partially classified sample arising from a two-component Weibull mixture.<n>The probability of a missing label is modelled as a function of classification uncertainty, giving a feature-dependent missing-at-random mechanism.<n>We show that a positive-definite increase in Fisher information is sufficient, but not necessary, for a smaller first-order expected error rate.
  arXiv  Detail & Related papers  (2026-09-01T06:06:53Z)
• Learning from Uncertainty-dependent Missing Labels for Semi-supervised Classification [2.5794915063815664]
  We study a semi-supervised setting in which the probability of label missingness depends on the observed features.<n>We develop a likelihood-based information theory for such uncertainty-dependent missing labels.
  arXiv  Detail & Related papers  (2026-08-25T01:40:47Z)
• Informative missingness and its implications in semi-supervised learning [2.5794915063815664]
  Semi-supervised learning (SSL) constructs classifiers using both labelled and unlabelled data.<n>This defines an incomplete-data problem, which statistically can be formulated within the likelihood framework for finite mixture models.<n> Modelling such informative missingness offers a coherent statistical framework that unifies likelihood-based inference with the behaviour of empirical SSL methods.
  arXiv  Detail & Related papers  (2025-12-04T02:26:56Z)
• Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical [66.57396042747706]
  Complementary-label learning is a weakly supervised learning problem. We propose a consistent approach that does not rely on the uniform distribution assumption. We find that complementary-label learning can be expressed as a set of negative-unlabeled binary classification problems.
  arXiv  Detail & Related papers  (2023-11-27T02:59:17Z)
• Multi-Label Noise Transition Matrix Estimation with Label Correlations: Theory and Algorithm [73.94839250910977]
  Noisy multi-label learning has garnered increasing attention due to the challenges posed by collecting large-scale accurate labels. The introduction of transition matrices can help model multi-label noise and enable the development of statistically consistent algorithms. We propose a novel estimator that leverages label correlations without the need for anchor points or precise fitting of noisy class posteriors.
  arXiv  Detail & Related papers  (2023-09-22T08:35:38Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Preserving Fine-Grain Feature Information in Classification via Entropic Regularization [10.358087436626391]
  We show that standard cross-entropy can lead to overfitting to coarse-related features. We introduce an entropy-based regularization to promote more diversity in the feature space of trained models.
  arXiv  Detail & Related papers  (2022-08-07T09:25:57Z)
• Learning from Multiple Unlabeled Datasets with Partial Risk Regularization [80.54710259664698]
  In this paper, we aim to learn an accurate classifier without any class labels. We first derive an unbiased estimator of the classification risk that can be estimated from the given unlabeled sets. We then find that the classifier obtained as such tends to cause overfitting as its empirical risks go negative during training. Experiments demonstrate that our method effectively mitigates overfitting and outperforms state-of-the-art methods for learning from multiple unlabeled sets.
  arXiv  Detail & Related papers  (2022-07-04T16:22:44Z)
• Multi-class Probabilistic Bounds for Self-learning [13.875239300089861]
  Pseudo-labeling is prone to error and runs the risk of adding noisy labels into unlabeled training data. We present a probabilistic framework for analyzing self-learning in the multi-class classification scenario with partially labeled data.
  arXiv  Detail & Related papers  (2021-09-29T13:57:37Z)
• Minimax Active Learning [61.729667575374606]
  Active learning aims to develop label-efficient algorithms by querying the most representative samples to be labeled by a human annotator. Current active learning techniques either rely on model uncertainty to select the most uncertain samples or use clustering or reconstruction to choose the most diverse set of unlabeled examples. We develop a semi-supervised minimax entropy-based active learning algorithm that leverages both uncertainty and diversity in an adversarial manner.
  arXiv  Detail & Related papers  (2020-12-18T19:03:40Z)
• Classification with Rejection Based on Cost-sensitive Classification [83.50402803131412]
  We propose a novel method of classification with rejection by ensemble of learning. Experimental results demonstrate the usefulness of our proposed approach in clean, noisy, and positive-unlabeled classification.
  arXiv  Detail & Related papers  (2020-10-22T14:05:05Z)
• Automatically Discovering and Learning New Visual Categories with Ranking Statistics [145.89790963544314]
  We tackle the problem of discovering novel classes in an image collection given labelled examples of other classes. We learn a general-purpose clustering model and use the latter to identify the new classes in the unlabelled data. We evaluate our approach on standard classification benchmarks and outperform current methods for novel category discovery by a significant margin.
  arXiv  Detail & Related papers  (2020-02-13T18:53:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.