Encoding categorical data: Is there yet anything 'hotter' than one-hot encoding?

• URL: http://arxiv.org/abs/2312.16930v1
• Date: Thu, 28 Dec 2023 09:56:44 GMT
• Title: Encoding categorical data: Is there yet anything 'hotter' than one-hot encoding?
• Abstract summary: The present study explores the encoding effects in an exhaustive sample of classification problems from OpenML repository. We found that in multiclass tasks, one-hot encoding and Helmert contrast coding outperform target-based encoders.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Categorical features are present in about 40% of real world problems, highlighting the crucial role of encoding as a preprocessing component. Some recent studies have reported benefits of the various target-based encoders over classical target-agnostic approaches. However, these claims are not supported by any statistical analysis, and are based on a single dataset or a very small and heterogeneous sample of datasets. The present study explores the encoding effects in an exhaustive sample of classification problems from OpenML repository. We fitted linear mixed-effects models to the experimental data, treating task ID as a random effect, and the encoding scheme and the various characteristics of categorical features as fixed effects. We found that in multiclass tasks, one-hot encoding and Helmert contrast coding outperform target-based encoders. In binary tasks, there were no significant differences across the encoding schemes; however, one-hot encoding demonstrated a marginally positive effect on the outcome. Importantly, we found no significant interactions between the encoding schemes and the characteristics of categorical features. This suggests that our findings are generalizable to a wide variety of problems across domains.

Related papers

• GATech at AbjadMed: Bidirectional Encoders vs. Causal Decoders: Insights from 82-Class Arabic Medical Classification [0.0]
  This paper presents system description for Arabic medical text classification across 82 distinct categories.<n>We benchmark this approach against a suite of multilingual and Arabic-specific encoders, as well as several large-scale causal decoders.<n> specialized bidirectional encoders significantly outperform causal decoders in capturing the precise semantic boundaries.
  arXiv  Detail & Related papers  (2026-02-17T16:13:08Z)
• Multi-Label Contrastive Learning : A Comprehensive Study [48.81069245141415]
  Multi-label classification has emerged as a key area in both research and industry. Applying contrastive learning to multi-label classification presents unique challenges. We conduct an in-depth study of contrastive learning loss for multi-label classification across diverse settings.
  arXiv  Detail & Related papers  (2024-11-27T20:20:06Z)
• Tabular Learning: Encoding for Entity and Context Embeddings [0.0]
  Examining the effect of different encoding techniques on entity and context embeddings. Applying different preprocessing methods and network architectures over several datasets resulted in a benchmark on how the encoders influence the learning outcome of the networks.
  arXiv  Detail & Related papers  (2024-03-28T13:29:29Z)
• Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
  Fine-grained image recognition is a longstanding computer vision challenge. We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Our method significantly improves the generalization performance on several popular classification networks.
  arXiv  Detail & Related papers  (2023-09-01T11:15:50Z)
• On the Suitability of Representations for Quality Diversity Optimization of Shapes [77.34726150561087]
  The representation, or encoding, utilized in evolutionary algorithms has a substantial effect on their performance. This study compares the impact of several representations, including direct encoding, a dictionary-based representation, parametric encoding, compositional pattern producing networks, and cellular automata, on the generation of voxelized meshes.
  arXiv  Detail & Related papers  (2023-04-07T07:34:23Z)
• Practical Approaches for Fair Learning with Multitype and Multivariate Sensitive Attributes [70.6326967720747]
  It is important to guarantee that machine learning algorithms deployed in the real world do not result in unfairness or unintended social consequences. We introduce FairCOCCO, a fairness measure built on cross-covariance operators on reproducing kernel Hilbert Spaces. We empirically demonstrate consistent improvements against state-of-the-art techniques in balancing predictive power and fairness on real-world datasets.
  arXiv  Detail & Related papers  (2022-11-11T11:28:46Z)
• Selecting the suitable resampling strategy for imbalanced data classification regarding dataset properties [62.997667081978825]
  In many application domains such as medicine, information retrieval, cybersecurity, social media, etc., datasets used for inducing classification models often have an unequal distribution of the instances of each class. This situation, known as imbalanced data classification, causes low predictive performance for the minority class examples. Oversampling and undersampling techniques are well-known strategies to deal with this problem by balancing the number of examples of each class.
  arXiv  Detail & Related papers  (2021-12-15T18:56:39Z)
• Quantile Encoder: Tackling High Cardinality Categorical Features in Regression Problems [2.3322477552758234]
  We provide an in-depth analysis of how to tackle high cardinality categor-ical features with the quantile. Our proposal outperforms state-of-the-encoders, including the traditional statistical mean target encoder. We also describe how to expand the encoded values by creating a set of features with different quantiles.
  arXiv  Detail & Related papers  (2021-05-27T11:56:13Z)
• Deep ensembles based on Stochastic Activation Selection for Polyp Segmentation [82.61182037130406]
  This work deals with medical image segmentation and in particular with accurate polyp detection and segmentation during colonoscopy examinations. Basic architecture in image segmentation consists of an encoder and a decoder. We compare some variant of the DeepLab architecture obtained by varying the decoder backbone.
  arXiv  Detail & Related papers  (2021-04-02T02:07:37Z)
• Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features [1.1709030738577393]
  We study techniques that yield numeric representations of categorical variables. We compare different encoding strategies together with five machine learning algorithms. Regularized versions of target encoding consistently provided the best results.
  arXiv  Detail & Related papers  (2021-04-01T17:21:42Z)
• Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorization [0.5570192369031546]
  hierarchical Poisson matrix factorization (HPF) and other sparse probabilistic non-negative matrix factorization (NMF) methods are considered to be interpretable generative models. HPF is often incorrectly interpreted in the literature as if it possesses encoder sparsity. We address this deficiency by self-consistently enforcing encoder sparsity, using a generalized additive model (GAM)
  arXiv  Detail & Related papers  (2020-12-08T02:27:22Z)
• Multi-scale Interactive Network for Salient Object Detection [91.43066633305662]
  We propose the aggregate interaction modules to integrate the features from adjacent levels. To obtain more efficient multi-scale features, the self-interaction modules are embedded in each decoder unit. Experimental results on five benchmark datasets demonstrate that the proposed method without any post-processing performs favorably against 23 state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-17T15:41:37Z)
• Establishing strong imputation performance of a denoising autoencoder in a wide range of missing data problems [0.0]
  We develop a consistent framework for both training and imputation. We benchmarked the results against state-of-the-art imputation methods. The developed autoencoder obtained the smallest error for all ranges of initial data corruption.
  arXiv  Detail & Related papers  (2020-04-06T12:00:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.