Classification Using Global and Local Mahalanobis Distances
- URL: http://arxiv.org/abs/2402.08283v1
- Date: Tue, 13 Feb 2024 08:22:42 GMT
- Title: Classification Using Global and Local Mahalanobis Distances
- Authors: Annesha Ghosh, Anil K. Ghosh, Rita SahaRay, and Soham Sarkar
- Abstract summary: We propose a novel semi-parametric classifier based on Mahalanobis distances of an observation from the competing classes.
Our tool is a generalized additive model with the logistic link function that uses these distances as features to estimate the posterior probabilities of the different classes.
- Score: 1.7811840395202345
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We propose a novel semi-parametric classifier based on Mahalanobis distances
of an observation from the competing classes. Our tool is a generalized
additive model with the logistic link function that uses these distances as
features to estimate the posterior probabilities of the different classes.
While popular parametric classifiers like linear and quadratic discriminant
analyses are mainly motivated by the normality of the underlying distributions,
the proposed classifier is more flexible and free from such parametric
assumptions. Since the densities of elliptic distributions are functions of
Mahalanobis distances, this classifier works well when the competing classes
are (nearly) elliptic. In such cases, it often outperforms popular
nonparametric classifiers, especially when the sample size is small compared to
the dimension of the data. To cope with non-elliptic and possibly multimodal
distributions, we propose a local version of the Mahalanobis distance.
Subsequently, we propose another classifier based on a generalized additive
model that uses the local Mahalanobis distances as features. This nonparametric
classifier usually performs like the Mahalanobis distance based semiparametric
classifier when the underlying distributions are elliptic, but outperforms it
for several non-elliptic and multimodal distributions. We also investigate the
behaviour of these two classifiers in high dimension, low sample size
situations. A thorough numerical study involving several simulated and real
datasets demonstrate the usefulness of the proposed classifiers in comparison
to many state-of-the-art methods.
Related papers
- FeCAM: Exploiting the Heterogeneity of Class Distributions in
Exemplar-Free Continual Learning [21.088762527081883]
Exemplar-free class-incremental learning (CIL) poses several challenges since it prohibits the rehearsal of data from previous tasks.
Recent approaches to incrementally learning the classifier by freezing the feature extractor after the first task have gained much attention.
We explore prototypical networks for CIL, which generate new class prototypes using the frozen feature extractor and classify the features based on the Euclidean distance to the prototypes.
arXiv Detail & Related papers (2023-09-25T11:54:33Z) - Counting Like Human: Anthropoid Crowd Counting on Modeling the
Similarity of Objects [92.80955339180119]
mainstream crowd counting methods regress density map and integrate it to obtain counting results.
Inspired by this, we propose a rational and anthropoid crowd counting framework.
arXiv Detail & Related papers (2022-12-02T07:00:53Z) - Parametric Classification for Generalized Category Discovery: A Baseline
Study [70.73212959385387]
Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples.
We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem.
We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
arXiv Detail & Related papers (2022-11-21T18:47:11Z) - Soft-margin classification of object manifolds [0.0]
A neural population responding to multiple appearances of a single object defines a manifold in the neural response space.
The ability to classify such manifold is of interest, as object recognition and other computational tasks require a response that is insensitive to variability within a manifold.
Soft-margin classifiers are a larger class of algorithms and provide an additional regularization parameter used in applications to optimize performance outside the training set.
arXiv Detail & Related papers (2022-03-14T12:23:36Z) - Learning Debiased and Disentangled Representations for Semantic
Segmentation [52.35766945827972]
We propose a model-agnostic and training scheme for semantic segmentation.
By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes.
Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
arXiv Detail & Related papers (2021-10-31T16:15:09Z) - Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
We take a two-step approach by first modeling the probability distribution and then sampling from that model.
We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
arXiv Detail & Related papers (2021-10-20T12:25:22Z) - Optimality and complexity of classification by random projection [1.5229257192293197]
The generalization error of a classifier is related to the complexity of the set of functions among which the classifier is chosen.
We show that this type of classifier is extremely flexible, as it is likely to approximate an arbitrary precision.
In particular, given full knowledge of the class conditional densities, the error of these low-complexity classifiers would converge to the optimal (Bayes) error as k and n go to infinity.
arXiv Detail & Related papers (2021-08-11T23:14:46Z) - Large scale analysis of generalization error in learning using margin
based classification methods [2.436681150766912]
We derive the expression for the generalization error of a family of large-margin classifiers in the limit of both sample size $n$ and dimension $p$.
For two layer neural networks, we reproduce the recently developed double descent' phenomenology for several classification models.
arXiv Detail & Related papers (2020-07-16T20:31:26Z) - Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers.
We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model.
Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
arXiv Detail & Related papers (2020-06-22T21:12:31Z) - Asymptotic Analysis of an Ensemble of Randomly Projected Linear
Discriminants [94.46276668068327]
In [1], an ensemble of randomly projected linear discriminants is used to classify datasets.
We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator.
We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
arXiv Detail & Related papers (2020-04-17T12:47:04Z) - On a Generalization of the Average Distance Classifier [2.578242050187029]
We propose some simple transformations of the average distance classifier to tackle this issue.
The resulting classifiers perform quite well even when the underlying populations have the same location and scale.
Numerical experiments with a variety of simulated as well as real data sets exhibit the usefulness of the proposed methodology.
arXiv Detail & Related papers (2020-01-08T10:00:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.