Which Imputation Fits Which Feature Selection Method? A Survey-Based Simulation Study

• URL: http://arxiv.org/abs/2412.13570v1
• Date: Wed, 18 Dec 2024 07:36:03 GMT
• Title: Which Imputation Fits Which Feature Selection Method? A Survey-Based Simulation Study
• Authors: Jakob Schwerter, Andrés Romero, Florian Dumpert, Markus Pauly,
• Abstract summary: Feature importance measures are usually considered for feature selection as well as to assess the effect of features on the outcome variables in the model. The typical solution is to impute the missing data before applying the learning method. We consider the two most common tree-based methods, Random Forest and XGBoost, and an interpretable linear model with regularization.
• Score: 4.335350817722218
• License:
• Abstract: Tree-based learning methods such as Random Forest and XGBoost are still the gold-standard prediction methods for tabular data. Feature importance measures are usually considered for feature selection as well as to assess the effect of features on the outcome variables in the model. This also applies to survey data, which are frequently encountered in the social sciences and official statistics. These types of datasets often present the challenge of missing values. The typical solution is to impute the missing data before applying the learning method. However, given the large number of possible imputation methods available, the question arises as to which should be chosen to achieve the 'best' reflection of feature importance and feature selection in subsequent analyses. In the present paper, we investigate this question in a survey-based simulation study for eight state-of-the art imputation methods and three learners. The imputation methods comprise listwise deletion, three MICE options, four \texttt{missRanger} options as well as the recently proposed mixGBoost imputation approach. As learners, we consider the two most common tree-based methods, Random Forest and XGBoost, and an interpretable linear model with regularization.

Related papers

• How to Select Datapoints for Efficient Human Evaluation of NLG Models? [57.60407340254572]
  We develop a suite of selectors to get the most informative datapoints for human evaluation. We show that selectors based on variance in automated metric scores, diversity in model outputs, or Item Response Theory outperform random selection. In particular, we introduce source-based estimators, which predict item usefulness for human evaluation just based on the source texts.
  arXiv  Detail & Related papers  (2025-01-30T10:33:26Z)
• Algorithm Selection with Probing Trajectories: Benchmarking the Choice of Classifier Model [0.20718016474717196]
  We conduct a benchmark study using 17 different classifiers and three types of trajectory on a classification task using the BBOB benchmark suite. We find that the choice of classifier has a significant impact, showing that feature-based and interval-based models are the best choices.
  arXiv  Detail & Related papers  (2025-01-20T11:28:45Z)
• Utilising Explainable Techniques for Quality Prediction in a Complex Textiles Manufacturing Use Case [0.0]
  This paper develops an approach to classify instances of product failure in a complex textiles manufacturing dataset using explainable techniques. In investigating the trade-off between accuracy and explainability, three different tree-based classification algorithms were evaluated.
  arXiv  Detail & Related papers  (2024-07-26T06:50:17Z)
• Feature Selection as Deep Sequential Generative Learning [50.00973409680637]
  We develop a deep variational transformer model over a joint of sequential reconstruction, variational, and performance evaluator losses. Our model can distill feature selection knowledge and learn a continuous embedding space to map feature selection decision sequences into embedding vectors associated with utility scores.
  arXiv  Detail & Related papers  (2024-03-06T16:31:56Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• Towards Free Data Selection with General-Purpose Models [71.92151210413374]
  A desirable data selection algorithm can efficiently choose the most informative samples to maximize the utility of limited annotation budgets. Current approaches, represented by active learning methods, typically follow a cumbersome pipeline that iterates the time-consuming model training and batch data selection repeatedly. FreeSel bypasses the heavy batch selection process, achieving a significant improvement in efficiency and being 530x faster than existing active learning methods.
  arXiv  Detail & Related papers  (2023-09-29T15:50:14Z)
• Lexicase Selection at Scale [5.4968949435821735]
  Lexicase selection is a semantic-aware parent selection method, which assesses individual test cases in a randomly-shuffled data stream. One potential drawback of lexicase selection and its variants is that the selection procedure requires evaluating training cases in a single data stream. We propose a novel method, fast lexicase selection, which incorporates lexicase selection and weighted shuffle with partial evaluation.
  arXiv  Detail & Related papers  (2022-08-23T03:58:47Z)
• Multivariate feature ranking of gene expression data [62.997667081978825]
  We propose two new multivariate feature ranking methods based on pairwise correlation and pairwise consistency. We statistically prove that the proposed methods outperform the state of the art feature ranking methods Clustering Variation, Chi Squared, Correlation, Information Gain, ReliefF and Significance.
  arXiv  Detail & Related papers  (2021-11-03T17:19:53Z)
• Variable selection with missing data in both covariates and outcomes: Imputation and machine learning [1.0333430439241666]
  The missing data issue is ubiquitous in health studies. Machine learning methods weaken parametric assumptions. XGBoost and BART have the overall best performance across various settings.
  arXiv  Detail & Related papers  (2021-04-06T20:18:29Z)
• Feature Selection Methods for Cost-Constrained Classification in Random Forests [3.4806267677524896]
  Cost-sensitive feature selection describes a feature selection problem, where features raise individual costs for inclusion in a model. Random Forests define a particularly challenging problem for feature selection, as features are generally entangled in an ensemble of multiple trees. We propose Shallow Tree Selection, a novel fast and multivariate feature selection method that selects features from small tree structures.
  arXiv  Detail & Related papers  (2020-08-14T11:39:52Z)
• Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior? [97.77183117452235]
  We carry out human subject tests to isolate the effect of algorithmic explanations on model interpretability. Clear evidence of method effectiveness is found in very few cases. Our results provide the first reliable and comprehensive estimates of how explanations influence simulatability.
  arXiv  Detail & Related papers  (2020-05-04T20:35:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.