On Clustering Categories of Categorical Predictors in Generalized Linear Models

• URL: http://arxiv.org/abs/2110.10059v1
• Date: Tue, 19 Oct 2021 15:36:35 GMT
• Title: On Clustering Categories of Categorical Predictors in Generalized Linear Models
• Authors: Emilio Carrizosa and Marcela Galvis Restrepo and Dolores Romero Morales
• Abstract summary: We propose a method to reduce the complexity of Generalized Linear Models in the presence of categorical predictors. The traditional one-hot encoding, where each category is represented by a dummy variable, can be wasteful, difficult to interpret, and prone to overfitting. This paper addresses these challenges by finding a reduced representation of the categorical predictors by clustering their categories.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We propose a method to reduce the complexity of Generalized Linear Models in the presence of categorical predictors. The traditional one-hot encoding, where each category is represented by a dummy variable, can be wasteful, difficult to interpret, and prone to overfitting, especially when dealing with high-cardinality categorical predictors. This paper addresses these challenges by finding a reduced representation of the categorical predictors by clustering their categories. This is done through a numerical method which aims to preserve (or even, improve) accuracy, while reducing the number of coefficients to be estimated for the categorical predictors. Thanks to its design, we are able to derive a proximity measure between categories of a categorical predictor that can be easily visualized. We illustrate the performance of our approach in real-world classification and count-data datasets where we see that clustering the categorical predictors reduces complexity substantially without harming accuracy.

Related papers

• Balanced Classification: A Unified Framework for Long-Tailed Object Detection [74.94216414011326]
  Conventional detectors suffer from performance degradation when dealing with long-tailed data due to a classification bias towards the majority head categories. We introduce a unified framework called BAlanced CLassification (BACL), which enables adaptive rectification of inequalities caused by disparities in category distribution. BACL consistently achieves performance improvements across various datasets with different backbones and architectures.
  arXiv  Detail & Related papers  (2023-08-04T09:11:07Z)
• A Statistical Model for Predicting Generalization in Few-Shot Classification [6.158812834002346]
  We introduce a Gaussian model of the feature distribution to predict the generalization error. We show that our approach outperforms alternatives such as the leave-one-out cross-validation strategy.
  arXiv  Detail & Related papers  (2022-12-13T10:21:15Z)
• Parametric Classification for Generalized Category Discovery: A Baseline Study [70.73212959385387]
  Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
  arXiv  Detail & Related papers  (2022-11-21T18:47:11Z)
• Generalization Error Bounds for Multiclass Sparse Linear Classifiers [7.360807642941714]
  We consider high-dimensional multiclass classification by sparse multinomial logistic regression. We propose a computationally feasible feature selection procedure based on penalized maximum likelihood. In particular, we consider global sparsity, double row-wise sparsity, and low-rank sparsity.
  arXiv  Detail & Related papers  (2022-04-13T09:25:03Z)
• Soft-margin classification of object manifolds [0.0]
  A neural population responding to multiple appearances of a single object defines a manifold in the neural response space. The ability to classify such manifold is of interest, as object recognition and other computational tasks require a response that is insensitive to variability within a manifold. Soft-margin classifiers are a larger class of algorithms and provide an additional regularization parameter used in applications to optimize performance outside the training set.
  arXiv  Detail & Related papers  (2022-03-14T12:23:36Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• DropLoss for Long-Tail Instance Segmentation [56.162929199998075]
  We develop DropLoss, a novel adaptive loss to compensate for the imbalance between rare and frequent categories. We show state-of-the-art mAP across rare, common, and frequent categories on the LVIS dataset.
  arXiv  Detail & Related papers  (2021-04-13T17:59:22Z)
• Asymptotic Supervised Predictive Classifiers under Partition Exchangeability [0.0]
  The result shows the convergence of these classifiers under infinite amount of training or test data. This is an important result from the practical perspective as under the presence of sufficiently large amount of data, one can replace the simpler marginal classifier with computationally more expensive simultaneous one.
  arXiv  Detail & Related papers  (2021-01-26T17:17:40Z)
• Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View [82.80085730891126]
  We provide the first modernally precise analysis of linear multiclass classification. Our analysis reveals that the classification accuracy is highly distribution-dependent. The insights gained may pave the way for a precise understanding of other classification algorithms.
  arXiv  Detail & Related papers  (2020-11-16T05:17:29Z)
• Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier [68.38233199030908]
  Long-tail recognition tackles the natural non-uniformly distributed data in realworld scenarios. While moderns perform well on populated classes, its performance degrades significantly on tail classes. Deep-RTC is proposed as a new solution to the long-tail problem, combining realism with hierarchical predictions.
  arXiv  Detail & Related papers  (2020-07-20T05:57:42Z)
• Deep Hierarchical Classification for Category Prediction in E-commerce System [16.6932395109085]
  In e-commerce system, category prediction is to automatically predict categories of given texts. We propose a Deep Hierarchical Classification framework, which incorporates the multi-scale hierarchical information in neural networks. We also define a novel combined loss function to punish hierarchical prediction losses.
  arXiv  Detail & Related papers  (2020-05-14T02:29:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.