Enumerating the k-fold configurations in multi-class classification problems

• URL: http://arxiv.org/abs/2401.13843v1
• Date: Wed, 24 Jan 2024 22:40:00 GMT
• Title: Enumerating the k-fold configurations in multi-class classification problems
• Authors: Attila Fazekas and Gyorgy Kovacs
• Abstract summary: The crisis faced by artificial intelligence partly results from the irreproducibility of reported k-fold cross-validation-based performance scores. Recently, we introduced numerical techniques to test the consistency of claimed performance scores and experimental setups. In a crucial use case, the method relies on the enumeration of all k-fold configurations, for which we proposed an algorithm in the binary classification case.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: K-fold cross-validation is a widely used tool for assessing classifier performance. The reproducibility crisis faced by artificial intelligence partly results from the irreproducibility of reported k-fold cross-validation-based performance scores. Recently, we introduced numerical techniques to test the consistency of claimed performance scores and experimental setups. In a crucial use case, the method relies on the combinatorial enumeration of all k-fold configurations, for which we proposed an algorithm in the binary classification case.

Related papers

• Binary Split Categorical feature with Mean Absolute Error Criteria in CART [18.476195198589462]
  Using the Mean Absolute Error criterion for categorical features has traditionally relied on various numerical encoding methods.<n>We present a novel and efficient splitting algorithm that addresses the challenges of handling categorical features with the MAE criterion.
  arXiv  Detail & Related papers  (2025-11-11T17:15:25Z)
• Interpretable Clustering with the Distinguishability Criterion [0.4419843514606336]
  We present a global criterion called the Distinguishability criterion to quantify the separability of identified clusters and validate inferred cluster configurations. We propose a combined loss function-based computational framework that integrates the Distinguishability criterion with many commonly used clustering procedures. We present these new algorithms as well as the results from comprehensive data analysis based on simulation studies and real data applications.
  arXiv  Detail & Related papers  (2024-04-24T16:38:15Z)
• Convolutional autoencoder-based multimodal one-class classification [80.52334952912808]
  One-class classification refers to approaches of learning using data from a single class only. We propose a deep learning one-class classification method suitable for multimodal data.
  arXiv  Detail & Related papers  (2023-09-25T12:31:18Z)
• Anomaly Detection using Ensemble Classification and Evidence Theory [62.997667081978825]
  We present a novel approach for novel detection using ensemble classification and evidence theory. A pool selection strategy is presented to build a solid ensemble classifier. We use uncertainty for the anomaly detection approach.
  arXiv  Detail & Related papers  (2022-12-23T00:50:41Z)
• The Impact of Using Regression Models to Build Defect Classifiers [13.840006058766766]
  It is common practice to discretize continuous defect counts into defective and non-defective classes. We compare the performance and interpretation of defect classifiers built using both approaches.
  arXiv  Detail & Related papers  (2022-02-12T22:12:55Z)
• Self-Certifying Classification by Linearized Deep Assignment [65.0100925582087]
  We propose a novel class of deep predictors for classifying metric data on graphs within PAC-Bayes risk certification paradigm. Building on the recent PAC-Bayes literature and data-dependent priors, this approach enables learning posterior distributions on the hypothesis space.
  arXiv  Detail & Related papers  (2022-01-26T19:59:14Z)
• When in Doubt: Improving Classification Performance with Alternating Normalization [57.39356691967766]
  We introduce Classification with Alternating Normalization (CAN), a non-parametric post-processing step for classification. CAN improves classification accuracy for challenging examples by re-adjusting their predicted class probability distribution. We empirically demonstrate its effectiveness across a diverse set of classification tasks.
  arXiv  Detail & Related papers  (2021-09-28T02:55:42Z)
• Re-Assessing the "Classify and Count" Quantification Method [88.60021378715636]
  "Classify and Count" (CC) is often a biased estimator. Previous works have failed to use properly optimised versions of CC. We argue that, while still inferior to some cutting-edge methods, they deliver near-state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2020-11-04T21:47:39Z)
• On the Efficiency of K-Means Clustering: Evaluation, Optimization, and Algorithm Selection [20.900296096958446]
  This paper presents a thorough evaluation of the existing methods that accelerate Lloyd's algorithm for fast k-means clustering. Within UniK, we thoroughly evaluate the pros and cons of existing methods using multiple performance metrics on a number of datasets. We derive an optimized algorithm over UniK, which effectively hybridizes multiple existing methods for more aggressive pruning.
  arXiv  Detail & Related papers  (2020-10-13T19:45:30Z)
• Machine Learning Clustering Techniques for Selective Mitigation of Critical Design Features [0.16311150636417257]
  This paper presents a new methodology which uses machine learning clustering techniques to group flip-flops with similar expected contributions to the overall functional failure rate. Fault simulation campaigns can then be executed on a per-group basis, significantly reducing the time and cost of the evaluation.
  arXiv  Detail & Related papers  (2020-08-31T15:03:16Z)
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing [105.91827623768724]
  Machine learning algorithms are susceptible to data poisoning attacks. We present a unifying view of randomized smoothing over arbitrary functions. We propose a new strategy for building classifiers that are pointwise-certifiably robust to general data poisoning attacks.
  arXiv  Detail & Related papers  (2020-02-07T21:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.