Bayesian Semi-supervised Multi-category Classification under Nonparanormality

• URL: http://arxiv.org/abs/2001.03798v3
• Date: Fri, 19 Jul 2024 00:41:43 GMT
• Title: Bayesian Semi-supervised Multi-category Classification under Nonparanormality
• Authors: Rui Zhu, Shuvrarghya Ghosh, Subhashis Ghosal,
• Abstract summary: Semi-supervised learning is a model training method that uses both labeled and unlabeled data. This paper proposes a fully Bayes semi-supervised learning algorithm that can be applied to any multi-category classification problem.
• Score: 2.307581190124002
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semi-supervised learning is a model training method that uses both labeled and unlabeled data. This paper proposes a fully Bayes semi-supervised learning algorithm that can be applied to any multi-category classification problem. We assume the labels are missing at random when using unlabeled data in a semi-supervised setting. Suppose we have $K$ classes in the data. We assume that the observations follow $K$ multivariate normal distributions depending on their true class labels after some common unknown transformation is applied to each component of the observation vector. The function is expanded in a B-splines series, and a prior is added to the coefficients. We consider a normal prior on the coefficients and constrain the values to meet the normality and identifiability constraints requirement. The precision matrices of the Gaussian distributions are given a conjugate Wishart prior, while the means are given the improper uniform prior. The resulting posterior is still conditionally conjugate, and the Gibbs sampler aided by a data-augmentation technique can thus be adopted. An extensive simulation study compares the proposed method with several other available methods. The proposed method is also applied to real datasets on diagnosing breast cancer and classification of signals. We conclude that the proposed method has a better prediction accuracy in various cases.

Related papers

• Benign Overfitting and the Geometry of the Ridge Regression Solution in Binary Classification [75.01389991485098]
  We show that ridge regression has qualitatively different behavior depending on the scale of the cluster mean vector. In regimes where the scale is very large, the conditions that allow for benign overfitting turn out to be the same as those for the regression task.
  arXiv  Detail & Related papers  (2025-03-11T01:45:42Z)
• Removing Geometric Bias in One-Class Anomaly Detection with Adaptive Feature Perturbation [10.556297392824721]
  One-class anomaly detection aims to detect objects that do not belong to a predefined normal class. We consider frozen yet rich feature spaces given by pretrained models and create pseudo-anomalous features with a novel adaptive linear feature perturbation technique. It adapts the noise distribution to each sample applies decaying linear perturbations to feature vectors and further guides the classification process using a contrastive learning objective.
  arXiv  Detail & Related papers  (2025-03-07T15:42:51Z)
• Class prior estimation for positive-unlabeled learning when label shift occurs [1.0514231683620516]
  We introduce a novel direct estimator of class prior which avoids estimation of posterior probabilities. It is based on a distribution matching technique together with kernel embedding and is obtained as an explicit solution to an optimisation task. We study finite sample behaviour for synthetic and real data and show that the proposal, together with a suitably modified version for large values of source prior, works on par or better than its competitors.
  arXiv  Detail & Related papers  (2025-02-28T16:12:53Z)
• A method for classification of data with uncertainty using hypothesis testing [0.0]
  It is necessary to quantify uncertainty and adopt decision-making approaches that take it into account. We propose a new decision-making approach using two types of hypothesis testing. This method is capable of detecting ambiguous data that belong to the overlapping regions of two class distributions.
  arXiv  Detail & Related papers  (2025-02-12T17:14:07Z)
• Concentration of Measure for Distributions Generated via Diffusion Models [16.868125342684603]
  We show via a combination of mathematical arguments and empirical evidence that data distributions sampled from diffusion models satisfy a Concentration of Measure Property. This implies that such models are quite restrictive and gives an explanation for a fact previously observed in the literature that conventional diffusion models cannot capture "heavy-tailed" data.
  arXiv  Detail & Related papers  (2025-01-13T23:13:01Z)
• Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification [49.09505771145326]
  We propose a Hierarchical Dynamic Labeling (HDL) algorithm that does not depend on model predictions and utilizes image embeddings to generate sample labels. Our approach has the potential to change the paradigm of pseudo-label generation in semi-supervised learning.
  arXiv  Detail & Related papers  (2024-04-26T06:00:27Z)
• Discriminative calibration: Check Bayesian computation from simulations and flexible classifier [23.91355980551754]
  We propose to replace the marginal rank test with a flexible classification approach that learns test statistics from data. We illustrate an automated implementation using neural networks and statistically-inspired features, and validate the method with numerical and real data experiments.
  arXiv  Detail & Related papers  (2023-05-24T00:18:48Z)
• Compound Batch Normalization for Long-tailed Image Classification [77.42829178064807]
  We propose a compound batch normalization method based on a Gaussian mixture. It can model the feature space more comprehensively and reduce the dominance of head classes. The proposed method outperforms existing methods on long-tailed image classification.
  arXiv  Detail & Related papers  (2022-12-02T07:31:39Z)
• Parametric Classification for Generalized Category Discovery: A Baseline Study [70.73212959385387]
  Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
  arXiv  Detail & Related papers  (2022-11-21T18:47:11Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• DenseHybrid: Hybrid Anomaly Detection for Dense Open-set Recognition [1.278093617645299]
  Anomaly detection can be conceived either through generative modelling of regular training data or by discriminating with respect to negative training data. This paper presents a novel hybrid anomaly score which allows dense open-set recognition on large natural images. Experiments evaluate our contributions on standard dense anomaly detection benchmarks as well as in terms of open-mIoU - a novel metric for dense open-set performance.
  arXiv  Detail & Related papers  (2022-07-06T11:48:50Z)
• Positive-Unlabeled Classification under Class-Prior Shift: A Prior-invariant Approach Based on Density Ratio Estimation [85.75352990739154]
  We propose a novel PU classification method based on density ratio estimation. A notable advantage of our proposed method is that it does not require the class-priors in the training phase.
  arXiv  Detail & Related papers  (2021-07-11T13:36:53Z)
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization [88.91872713134342]
  We propose a theoretically grounded method that can estimate the noise transition matrix and learn a classifier simultaneously. We show the effectiveness of the proposed method through experiments on benchmark and real-world datasets.
  arXiv  Detail & Related papers  (2021-02-04T05:09:18Z)
• Large scale analysis of generalization error in learning using margin based classification methods [2.436681150766912]
  We derive the expression for the generalization error of a family of large-margin classifiers in the limit of both sample size $n$ and dimension $p$. For two layer neural networks, we reproduce the recently developed double descent' phenomenology for several classification models.
  arXiv  Detail & Related papers  (2020-07-16T20:31:26Z)
• Handling missing data in model-based clustering [0.0]
  We propose two methods to fit Gaussian mixtures in the presence of missing data. Both methods use a variant of the Monte Carlo Expectation-Maximisation algorithm for data augmentation. We show that the proposed methods outperform the multiple imputation approach, both in terms of clusters identification and density estimation.
  arXiv  Detail & Related papers  (2020-06-04T15:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.