From Theory to Practice: Implementing and Evaluating e-Fold Cross-Validation

• URL: http://arxiv.org/abs/2410.09463v2
• Date: Sat, 26 Oct 2024 18:25:35 GMT
• Title: From Theory to Practice: Implementing and Evaluating e-Fold Cross-Validation
• Authors: Christopher Mahlich, Tobias Vente, Joeran Beel,
• Abstract summary: e-fold cross-validation is an energy-efficient alternative to k-fold cross-validation. It requires 4 fewer folds than 10-fold cross-validation, reducing evaluation time, computational resources, and energy use by about 40%. E-fold cross-validation offers a reliable and efficient alternative to k-fold, reducing computational costs while maintaining comparable accuracy.
• Score: 0.10241134756773229
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces e-fold cross-validation, an energy-efficient alternative to k-fold cross-validation. It dynamically adjusts the number of folds based on a stopping criterion. The criterion checks after each fold whether the standard deviation of the evaluated folds has consistently decreased or remained stable. Once met, the process stops early. We tested e-fold cross-validation on 15 datasets and 10 machine-learning algorithms. On average, it required 4 fewer folds than 10-fold cross-validation, reducing evaluation time, computational resources, and energy use by about 40%. Performance differences between e-fold and 10-fold cross-validation were less than 2% for larger datasets. More complex models showed even smaller discrepancies. In 96% of iterations, the results were within the confidence interval, confirming statistical significance. E-fold cross-validation offers a reliable and efficient alternative to k-fold, reducing computational costs while maintaining comparable accuracy.

Related papers

• The Relative Instability of Model Comparison with Cross-validation [65.90853456199493]
  Cross-validation can be used to provide a confidence interval for the test error of a stable machine learning algorithm.<n>Relative stability cannot easily be derived from existing stability results, even for simple algorithms.<n>We empirically confirm the invalidity of CV confidence intervals for the test error difference when either soft-thresholding or the Lasso is used.
  arXiv  Detail & Related papers  (2025-08-06T12:54:56Z)
• Irredundant k-Fold Cross-Validation [0.0]
  In traditional k-fold cross-validation, each instance is used ($k!-!1$) times for training and once for testing, leading to redundancy.<n>We introduce Irredundant $k$--fold cross-validation, a novel method that guarantees each instance is used exactly once for training and once for testing.
  arXiv  Detail & Related papers  (2025-07-26T19:59:37Z)
• Stability Regularized Cross-Validation [5.156484100374059]
  We revisit the problem of ensuring strong test-set performance via cross-validation.<n>Motivated by the generalization theory literature, we propose a nested k-fold cross-validation scheme.<n>We benchmark our procedure on a suite of 13 real-world UCI datasets.
  arXiv  Detail & Related papers  (2025-05-11T10:06:59Z)
• A New Flexible Train-Test Split Algorithm, an approach for choosing among the Hold-out, K-fold cross-validation, and Hold-out iteration [0.0]
  This study focuses on improving the accuracy of ML algorithms across three different datasets. By modifying parameters like test size, Random State, and 'k' values, we were able to improve accuracy assessment. This study challenges the universality of K values in K-Fold Cross Validation and suggests a 10% test size and 90% training size for better outcomes.
  arXiv  Detail & Related papers  (2025-01-11T09:42:13Z)
• e-Fold Cross-Validation for Recommender-System Evaluation [0.09374652839580183]
  e-fold cross validation aims to minimize the number of folds to achieve a reduction in power usage. On average e-fold cross validation only needed 41.5% of the energy that 10-fold cross validation would need, while it's results only differed by 1.81%.
  arXiv  Detail & Related papers  (2024-12-02T00:05:20Z)
• Don't Waste Your Time: Early Stopping Cross-Validation [41.092016771160566]
  Cross-validation drastically increases the computational cost of validating a single configuration. Our study shows that even a simple-to-understand and easy-to-implement method consistently allows model selection to converge faster.
  arXiv  Detail & Related papers  (2024-05-06T11:51:09Z)
• Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Data [13.069717985067937]
  We propose a weighted rolling-validation procedure, an online variant of leave-one-out cross-validation. Similar to batch cross-validation, it can boost base estimators to achieve a better, adaptive convergence rate.
  arXiv  Detail & Related papers  (2023-10-18T17:52:57Z)
• Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networks [109.59988683444986]
  MeanQ is a simple ensemble method that estimates target values as ensemble means. We show that MeanQ shows remarkable sample efficiency in experiments on the Atari Learning Environment benchmark.
  arXiv  Detail & Related papers  (2022-09-16T01:47:36Z)
• Fast and Informative Model Selection using Learning Curve Cross-Validation [2.28438857884398]
  Cross-validation methods can be unnecessarily slow on large datasets. We present a new approach for validation based on learning curves (LCCV) LCCV iteratively increases the number of instances used for training.
  arXiv  Detail & Related papers  (2021-11-27T14:48:52Z)
• VSAC: Efficient and Accurate Estimator for H and F [68.65610177368617]
  VSAC is a RANSAC-type robust estimator with a number of novelties. It is significantly faster than all its predecessors and runs on average in 1-2 ms, on a CPU. It is two orders of magnitude faster and yet as precise as MAGSAC++, the currently most accurate estimator of two-view geometry.
  arXiv  Detail & Related papers  (2021-06-18T17:04:57Z)
• Fast calculation of Gaussian Process multiple-fold cross-validation residuals and their covariances [0.6091702876917281]
  We generalize fast leave-one-out formulae to multiple-fold cross-validation. We highlight the covariance structure of cross-validation residuals in both Simple and Universal Kriging frameworks. Our results enable fast multiple-fold cross-validation and have direct consequences in model diagnostics.
  arXiv  Detail & Related papers  (2021-01-08T17:02:37Z)
• Cross-validation Confidence Intervals for Test Error [83.67415139421448]
  This work develops central limit theorems for crossvalidation and consistent estimators of its variance under weak stability conditions on the learning algorithm. Results are the first of their kind for the popular choice of leave-one-out cross-validation.
  arXiv  Detail & Related papers  (2020-07-24T17:40:06Z)
• Second-Order Provable Defenses against Adversarial Attacks [63.34032156196848]
  We show that if the eigenvalues of the network are bounded, we can compute a certificate in the $l$ norm efficiently using convex optimization. We achieve certified accuracy of 5.78%, and 44.96%, and 43.19% on 2,59% and 4BP-based methods respectively.
  arXiv  Detail & Related papers  (2020-06-01T05:55:18Z)
• Towards new cross-validation-based estimators for Gaussian process regression: efficient adjoint computation of gradients [0.0]
  We suggest using new cross-validation criteria derived from the literature of scoring rules. We also provide an efficient method for computing the gradient of a cross-validation criterion.
  arXiv  Detail & Related papers  (2020-02-26T14:50:54Z)
• Cross-Iteration Batch Normalization [67.83430009388678]
  We present Cross-It Batch Normalization (CBN), in which examples from multiple recent iterations are jointly utilized to enhance estimation quality. CBN is found to outperform the original batch normalization and a direct calculation of statistics over previous iterations without the proposed compensation technique.
  arXiv  Detail & Related papers  (2020-02-13T18:52:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.