Lexicase Selection Parameter Analysis: Varying Population Size and Test Case Redundancy with Diagnostic Metrics

• URL: http://arxiv.org/abs/2407.15056v1
• Date: Sun, 21 Jul 2024 04:54:27 GMT
• Title: Lexicase Selection Parameter Analysis: Varying Population Size and Test Case Redundancy with Diagnostic Metrics
• Authors: Jose Guadalupe Hernandez, Anil Kumar Saini, Jason H. Moore,
• Abstract summary: We study how population size and number of generations affect lexicase's ability to exploit gradients and maintain specialists. By varying the population size with a fixed evaluation budget, we show that smaller populations tend to have greater exploitation capabilities. We also consider the effect redundant test cases have on specialist maintenance, and find that high redundancy may hinder the ability to optimize and maintain specialists.
• Score: 1.9570926122713395
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lexicase selection is a successful parent selection method in genetic programming that has outperformed other methods across multiple benchmark suites. Unlike other selection methods that require explicit parameters to function, such as tournament size in tournament selection, lexicase selection does not. However, if evolutionary parameters like population size and number of generations affect the effectiveness of a selection method, then lexicase's performance may also be impacted by these `hidden' parameters. Here, we study how these hidden parameters affect lexicase's ability to exploit gradients and maintain specialists using diagnostic metrics. By varying the population size with a fixed evaluation budget, we show that smaller populations tend to have greater exploitation capabilities, whereas larger populations tend to maintain more specialists. We also consider the effect redundant test cases have on specialist maintenance, and find that high redundancy may hinder the ability to optimize and maintain specialists, even for larger populations. Ultimately, we highlight that population size, evaluation budget, and test cases must be carefully considered for the characteristics of the problem being solved.

Related papers

• On the Robustness of Lexicase Selection to Contradictory Objectives [0.9208007322096533]
  We study lexicase and epsilon-lexicase selection's performance on contradictory objectives. We find that lexicase and epsilon-lexicase selection each have a region of parameter space where they are incapable of optimizing contradictory objectives. We propose theoretically-backed guidelines for parameter choice.
  arXiv  Detail & Related papers  (2024-03-11T15:23:35Z)
• Unified Uncertainty Estimation for Cognitive Diagnosis Models [70.46998436898205]
  We propose a unified uncertainty estimation approach for a wide range of cognitive diagnosis models. We decompose the uncertainty of diagnostic parameters into data aspect and model aspect. Our method is effective and can provide useful insights into the uncertainty of cognitive diagnosis.
  arXiv  Detail & Related papers  (2024-03-09T13:48:20Z)
• Feature Selection as Deep Sequential Generative Learning [50.00973409680637]
  We develop a deep variational transformer model over a joint of sequential reconstruction, variational, and performance evaluator losses. Our model can distill feature selection knowledge and learn a continuous embedding space to map feature selection decision sequences into embedding vectors associated with utility scores.
  arXiv  Detail & Related papers  (2024-03-06T16:31:56Z)
• Improving Recommendation System Serendipity Through Lexicase Selection [53.57498970940369]
  We propose a new serendipity metric to measure the presence of echo chambers and homophily in recommendation systems. We then attempt to improve the diversity-preservation qualities of well known recommendation techniques by adopting a parent selection algorithm known as lexicase selection. Our results show that lexicase selection, or a mixture of lexicase selection and ranking, outperforms its purely ranked counterparts in terms of personalization, coverage and our specifically designed serendipity benchmark.
  arXiv  Detail & Related papers  (2023-05-18T15:37:38Z)
• In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation [92.51773744318119]
  This paper empirically investigates the strengths and weaknesses of different model selection criteria. We highlight that there is a complex interplay between selection strategies, candidate estimators and the data used for comparing them.
  arXiv  Detail & Related papers  (2023-02-06T16:55:37Z)
• Lexicase Selection at Scale [5.4968949435821735]
  Lexicase selection is a semantic-aware parent selection method, which assesses individual test cases in a randomly-shuffled data stream. One potential drawback of lexicase selection and its variants is that the selection procedure requires evaluating training cases in a single data stream. We propose a novel method, fast lexicase selection, which incorporates lexicase selection and weighted shuffle with partial evaluation.
  arXiv  Detail & Related papers  (2022-08-23T03:58:47Z)
• Flexible variable selection in the presence of missing data [0.0]
  We propose a non-parametric variable selection algorithm combined with multiple imputation to develop flexible panels in the presence of missing-at-random data. We show that our proposal has good operating characteristics and results in panels with higher classification and variable selection performance.
  arXiv  Detail & Related papers  (2022-02-25T21:41:03Z)
• Compactness Score: A Fast Filter Method for Unsupervised Feature Selection [66.84571085643928]
  We propose a fast unsupervised feature selection method, named as, Compactness Score (CSUFS) to select desired features. Our proposed algorithm seems to be more accurate and efficient compared with existing algorithms.
  arXiv  Detail & Related papers  (2022-01-31T13:01:37Z)
• An Exploration of Exploration: Measuring the ability of lexicase selection to find obscure pathways to optimality [62.997667081978825]
  We introduce an "exploration diagnostic" that diagnoses a selection scheme's capacity for search space exploration. We verify that lexicase selection out-explores tournament selection. We find that relaxing lexicase's elitism with epsilon lexicase can further improve exploration.
  arXiv  Detail & Related papers  (2021-07-20T20:43:06Z)
• Problem-solving benefits of down-sampled lexicase selection [0.20305676256390928]
  We show that down-sampled lexicase selection's main benefit stems from the fact that it allows the evolutionary process to examine more individuals within the same computational budget. The reasons that down-sampling helps, however, are not yet fully understood.
  arXiv  Detail & Related papers  (2021-06-10T23:42:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.