Minimally Informed Linear Discriminant Analysis: training an LDA model with unlabelled data

• URL: http://arxiv.org/abs/2310.11110v1
• Date: Tue, 17 Oct 2023 09:50:31 GMT
• Title: Minimally Informed Linear Discriminant Analysis: training an LDA model with unlabelled data
• Authors: Nicolas Heintz, Tom Francart, Alexander Bertrand
• Abstract summary: We show that it is possible to compute the exact projection vector from LDA models based on unlabelled data. We show that the MILDA projection vector can be computed in a closed form with a computational cost comparable to LDA.
• Score: 51.673443581397954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Linear Discriminant Analysis (LDA) is one of the oldest and most popular linear methods for supervised classification problems. In this paper, we demonstrate that it is possible to compute the exact projection vector from LDA models based on unlabelled data, if some minimal prior information is available. More precisely, we show that only one of the following three pieces of information is actually sufficient to compute the LDA projection vector if only unlabelled data are available: (1) the class average of one of the two classes, (2) the difference between both class averages (up to a scaling), or (3) the class covariance matrices (up to a scaling). These theoretical results are validated in numerical experiments, demonstrating that this minimally informed Linear Discriminant Analysis (MILDA) model closely matches the performance of a supervised LDA model. Furthermore, we show that the MILDA projection vector can be computed in a closed form with a computational cost comparable to LDA and is able to quickly adapt to non-stationary data, making it well-suited to use as an adaptive classifier.

Related papers

• GO-LDA: Generalised Optimal Linear Discriminant Analysis [6.644357197885522]
  Linear discriminant analysis has been a useful tool in pattern recognition and data analysis research and practice. We show that the generalised eigenanalysis solution to multiclass LDA does neither yield orthogonal discriminant directions nor maximise discrimination of projected data along them.
  arXiv  Detail & Related papers  (2023-05-23T23:11:05Z)
• Sketched Gaussian Model Linear Discriminant Analysis via the Randomized Kaczmarz Method [7.593861427248019]
  We present sketched linear discriminant analysis, an iterative randomized approach to binary-class Gaussian model linear discriminant analysis (LDA) for very large data. We harness a least squares formulation and mobilize the descent gradient framework. We present convergence guarantees for the sketched predictions on new data within a fixed number of iterations.
  arXiv  Detail & Related papers  (2022-11-10T18:29:36Z)
• Domain Adaptation Principal Component Analysis: base linear method for learning with out-of-distribution data [55.41644538483948]
  Domain adaptation is a popular paradigm in modern machine learning. We present a method called Domain Adaptation Principal Component Analysis (DAPCA) DAPCA finds a linear reduced data representation useful for solving the domain adaptation task.
  arXiv  Detail & Related papers  (2022-08-28T21:10:56Z)
• Revisiting Classical Multiclass Linear Discriminant Analysis with a Novel Prototype-based Interpretable Solution [0.0]
  We introduce a novel solution to classical LDA, called LDA++, that yields $C$ features, each one interpretable as measuring similarity to one cluster. This novel solution bridges between dimensionality reduction and multiclass classification.
  arXiv  Detail & Related papers  (2022-05-02T06:12:42Z)
• Weight Vector Tuning and Asymptotic Analysis of Binary Linear Classifiers [82.5915112474988]
  This paper proposes weight vector tuning of a generic binary linear classifier through the parameterization of a decomposition of the discriminant by a scalar. It is also found that weight vector tuning significantly improves the performance of Linear Discriminant Analysis (LDA) under high estimation noise.
  arXiv  Detail & Related papers  (2021-10-01T17:50:46Z)
• Self-Weighted Robust LDA for Multiclass Classification with Edge Classes [111.5515086563592]
  A novel self-weighted robust LDA with l21-norm based between-class distance criterion, called SWRLDA, is proposed for multi-class classification. The proposed SWRLDA is easy to implement, and converges fast in practice.
  arXiv  Detail & Related papers  (2020-09-24T12:32:55Z)
• High-Dimensional Quadratic Discriminant Analysis under Spiked Covariance Model [101.74172837046382]
  We propose a novel quadratic classification technique, the parameters of which are chosen such that the fisher-discriminant ratio is maximized. Numerical simulations show that the proposed classifier not only outperforms the classical R-QDA for both synthetic and real data but also requires lower computational complexity.
  arXiv  Detail & Related papers  (2020-06-25T12:00:26Z)
• A Doubly Regularized Linear Discriminant Analysis Classifier with Automatic Parameter Selection [24.027886914804775]
  Linear discriminant analysis (LDA) based classifiers tend to falter in many practical settings where the training data size is smaller than, or comparable to, the number of features. We propose a doubly regularized LDA classifier that we denote as R2LDA. Results obtained from both synthetic and real data demonstrate the consistency and effectiveness of the proposed R2LDA approach.
  arXiv  Detail & Related papers  (2020-04-28T07:09:22Z)
• Saliency-based Weighted Multi-label Linear Discriminant Analysis [101.12909759844946]
  We propose a new variant of Linear Discriminant Analysis (LDA) to solve multi-label classification tasks. The proposed method is based on a probabilistic model for defining the weights of individual samples. The Saliency-based weighted Multi-label LDA approach is shown to lead to performance improvements in various multi-label classification problems.
  arXiv  Detail & Related papers  (2020-04-08T19:40:53Z)
• Improving Covariance-Regularized Discriminant Analysis for EHR-based Predictive Analytics of Diseases [20.697847129363463]
  We study an analytical model that understands the accuracy of LDA for classifying data with arbitrary distribution. We also propose a novel LDA classifier De-Sparse that outperforms state-of-the-art LDA approaches developed for HDLSS data.
  arXiv  Detail & Related papers  (2016-10-18T06:11:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.