Greedy feature selection: Classifier-dependent feature selection via greedy methods

• URL: http://arxiv.org/abs/2403.05138v1
• Date: Fri, 8 Mar 2024 08:12:05 GMT
• Title: Greedy feature selection: Classifier-dependent feature selection via greedy methods
• Authors: Fabiana Camattari, Sabrina Guastavino, Francesco Marchetti, Michele Piana, Emma Perracchione
• Abstract summary: The purpose of this study is to introduce a new approach to feature ranking for classification tasks, called in what follows greedy feature selection. The benefits of such scheme are investigated theoretically in terms of model capacity indicators, such as the Vapnik-Chervonenkis (VC) dimension or the kernel alignment.
• Score: 2.4374097382908477
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The purpose of this study is to introduce a new approach to feature ranking for classification tasks, called in what follows greedy feature selection. In statistical learning, feature selection is usually realized by means of methods that are independent of the classifier applied to perform the prediction using that reduced number of features. Instead, greedy feature selection identifies the most important feature at each step and according to the selected classifier. In the paper, the benefits of such scheme are investigated theoretically in terms of model capacity indicators, such as the Vapnik-Chervonenkis (VC) dimension or the kernel alignment, and tested numerically by considering its application to the problem of predicting geo-effective manifestations of the active Sun.

Related papers

• Feature Selection as Deep Sequential Generative Learning [50.00973409680637]
  We develop a deep variational transformer model over a joint of sequential reconstruction, variational, and performance evaluator losses. Our model can distill feature selection knowledge and learn a continuous embedding space to map feature selection decision sequences into embedding vectors associated with utility scores.
  arXiv  Detail & Related papers  (2024-03-06T16:31:56Z)
• A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning [131.2910403490434]
  Data scientists typically collect as many features as possible into their datasets, and even engineer new features from existing ones. Existing benchmarks for tabular feature selection consider classical downstream models, toy synthetic datasets, or do not evaluate feature selectors on the basis of downstream performance. We construct a challenging feature selection benchmark evaluated on downstream neural networks including transformers. We also propose an input-gradient-based analogue of Lasso for neural networks that outperforms classical feature selection methods on challenging problems.
  arXiv  Detail & Related papers  (2023-11-10T05:26:10Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• Sequential Attention for Feature Selection [12.89764845700709]
  We propose a feature selection algorithm called Sequential Attention that achieves state-of-the-art empirical results for neural networks. We give theoretical insights into our algorithm for linear regression by showing that an adaptation to this setting is equivalent to the classical Orthogonal Matching Pursuit (OMP) algorithm.
  arXiv  Detail & Related papers  (2022-09-29T15:49:06Z)
• Handcrafted Feature Selection Techniques for Pattern Recognition: A Survey [0.0]
  Feature selection is a process that allows for representing information properly. This paper presents a survey on some Filters and Wrapper methods for handcrafted feature selection.
  arXiv  Detail & Related papers  (2022-09-06T18:05:35Z)
• Stop Overcomplicating Selective Classification: Use Max-Logit [2.3677503557659705]
  We tackle the problem of Selective Classification where the goal is to achieve the best performance on the desired coverages of the dataset. Recent state-of-the-art selective methods come with architectural changes either via introducing a separate selection head or an extra abstention logit. We present surprising results for Selective Classification by confirming that the superior performance of state-of-the-art methods is owed to training a more generalizable classifier.
  arXiv  Detail & Related papers  (2022-06-17T22:23:11Z)
• Compactness Score: A Fast Filter Method for Unsupervised Feature Selection [66.84571085643928]
  We propose a fast unsupervised feature selection method, named as, Compactness Score (CSUFS) to select desired features. Our proposed algorithm seems to be more accurate and efficient compared with existing algorithms.
  arXiv  Detail & Related papers  (2022-01-31T13:01:37Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• Dynamic Instance-Wise Classification in Correlated Feature Spaces [15.351282873821935]
  In a typical machine learning setting, the predictions on all test instances are based on a common subset of features discovered during model training. A new method is proposed that sequentially selects the best feature to evaluate for each test instance individually, and stops the selection process to make a prediction once it determines that no further improvement can be achieved with respect to classification accuracy. The effectiveness, generalizability, and scalability of the proposed method is illustrated on a variety of real-world datasets from diverse application domains.
  arXiv  Detail & Related papers  (2021-06-08T20:20:36Z)
• Deep Learning feature selection to unhide demographic recommender systems factors [63.732639864601914]
  The matrix factorization model generates factors which do not incorporate semantic knowledge. DeepUnHide is able to extract demographic information from the users and items factors in collaborative filtering recommender systems.
  arXiv  Detail & Related papers  (2020-06-17T17:36:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.