A Comparative Study of Model Selection Criteria for Symbolic Regression

• URL: http://arxiv.org/abs/2605.11233v1
• Date: Mon, 11 May 2026 20:47:50 GMT
• Title: A Comparative Study of Model Selection Criteria for Symbolic Regression
• Abstract summary: We rank candidate expressions generated by perturbing ground-truth functions to assess generalization error and selection probability of the ground-truth expression.<n>Our findings reveal that MDL consistently identifies models with the lowest test error and the shortest length across most datasets.
• Score: 37.31373989245858
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effective model selection is critical in symbolic regression (SR) to identify mathematical expressions that balance accuracy and complexity, and have low expected error on unseen data. Many modern implementations of genetic programming (GP) for SR generate a set of Pareto optimal candidate solutions, but reliable automatic selection of solutions that generalize well remains an open issue. Current literature offers various information-theoretic and Bayesian approaches, yet comprehensive comparisons of their performance across different data regimes are limited. This study presents a systematic empirical comparison of widely used selection criteria: the Akaike information criterion (AIC), the corrected AIC (AICc), the Bayesian information criterion (BIC), minimum description length (MDL), as well as Efron's bootstrap estimate for the in-sample prediction error on seven synthetic datasets with Gaussian noise. We rank candidate expressions generated by perturbing ground-truth functions to assess generalization error and selection probability of the ground-truth expression. Our findings reveal that MDL consistently identifies models with the lowest test error and the shortest length across most datasets. While no single criterion dominates all results, MDL and BIC produced the highest probability of selecting the ground-truth expressions.

Related papers

• A Mathematical Optimization Approach for Expert-Informed Bayesian Best Subset Selection [0.0]
  Expert-Implied Bayesian Best Subsets (EBBS) is a method that incorporates domain-expert probability estimates of feature relevance into the MIO best-subsets problem.<n>This paper provides analytic derivations of the MAP formulation and characterizes its theoretical properties.
  arXiv  Detail & Related papers  (2026-06-28T17:26:41Z)
• Consistency-based Abductive Reasoning over Perceptual Errors of Multiple Pre-trained Models in Novel Environments [5.5855749614100825]
  This paper addresses the hypothesis that leveraging multiple pre-trained models can mitigate this recall reduction.<n>We formulate the challenge of identifying and managing conflicting predictions from various models as a consistency-based abduction problem.<n>Our results validate the use of consistency-based abduction as an effective mechanism to robustly integrate knowledge from multiple imperfect models in challenging, novel scenarios.
  arXiv  Detail & Related papers  (2025-05-25T23:17:47Z)
• Minimum variance threshold for epsilon-lexicase selection [0.7373617024876725]
  Methods often rely on average error over the entire dataset as a criterion to select the parents. We propose a new criteria that splits errors into two partitions that minimize the total variance within partitions. Our results show a better performance of our approach compared to traditional epsilon-lexicase selection in the real-world datasets.
  arXiv  Detail & Related papers  (2024-04-08T23:47:26Z)
• Maximum Likelihood Estimation is All You Need for Well-Specified Covariate Shift [34.414261291690856]
  Key challenge of modern machine learning systems is to achieve Out-of-Distribution (OOD) generalization. We show that classical Maximum Likelihood Estimation (MLE) purely using source data achieves the minimax optimality. We illustrate the wide applicability of our framework by instantiating it to three concrete examples.
  arXiv  Detail & Related papers  (2023-11-27T16:06:48Z)
• BRIO: Bringing Order to Abstractive Summarization [107.97378285293507]
  We propose a novel training paradigm which assumes a non-deterministic distribution. Our method achieves a new state-of-the-art result on the CNN/DailyMail (47.78 ROUGE-1) and XSum (49.07 ROUGE-1) datasets.
  arXiv  Detail & Related papers  (2022-03-31T05:19:38Z)
• Selecting the suitable resampling strategy for imbalanced data classification regarding dataset properties [62.997667081978825]
  In many application domains such as medicine, information retrieval, cybersecurity, social media, etc., datasets used for inducing classification models often have an unequal distribution of the instances of each class. This situation, known as imbalanced data classification, causes low predictive performance for the minority class examples. Oversampling and undersampling techniques are well-known strategies to deal with this problem by balancing the number of examples of each class.
  arXiv  Detail & Related papers  (2021-12-15T18:56:39Z)
• Evaluating State-of-the-Art Classification Models Against Bayes Optimality [106.50867011164584]
  We show that we can compute the exact Bayes error of generative models learned using normalizing flows. We use our approach to conduct a thorough investigation of state-of-the-art classification models.
  arXiv  Detail & Related papers  (2021-06-07T06:21:20Z)
• Characterizing Fairness Over the Set of Good Models Under Selective Labels [69.64662540443162]
  We develop a framework for characterizing predictive fairness properties over the set of models that deliver similar overall performance. We provide tractable algorithms to compute the range of attainable group-level predictive disparities. We extend our framework to address the empirically relevant challenge of selectively labelled data.
  arXiv  Detail & Related papers  (2021-01-02T02:11:37Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)
• A Compressive Classification Framework for High-Dimensional Data [12.284934135116515]
  We propose a compressive classification framework for settings where the data dimensionality is significantly higher than the sample size. The proposed method, referred to as regularized discriminant analysis (CRDA), is based on linear discriminant analysis. It has the ability to select significant features by using joint-sparsity promoting hard thresholding in the discriminant rule.
  arXiv  Detail & Related papers  (2020-05-09T06:55:00Z)
• Robust Generalised Quadratic Discriminant Analysis [6.308539010172309]
  The classification rule in GQDA is based on the sample mean vector and the sample dispersion matrix of a training sample, which are extremely non-robust under data contamination. The present paper investigates the performance of the GQDA classifier when the classical estimators of the mean vector and the dispersion matrix used therein are replaced by various robust counterparts.
  arXiv  Detail & Related papers  (2020-04-11T18:21:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.