EasyFS: an Efficient Model-free Feature Selection Framework via Elastic Transformation of Features

• URL: http://arxiv.org/abs/2402.05954v1
• Date: Sun, 4 Feb 2024 09:25:07 GMT
• Title: EasyFS: an Efficient Model-free Feature Selection Framework via Elastic Transformation of Features
• Authors: Jianming Lv, Sijun Xia, Depin Liang, Wei Chen
• Abstract summary: We propose an efficient model-free feature selection framework via elastic expansion and compression of features, namely EasyFS, to achieve better performance than state-of-the-art model-aware methods. Experiments on 21 different datasets show that EasyFS outperforms state-of-the-art methods up to 10.9% in the regression tasks and 5.7% in the classification tasks while saving more than 94% of the time.
• Score: 8.503238425293754
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional model-free feature selection methods treat each feature independently while disregarding the interrelationships among features, which leads to relatively poor performance compared with the model-aware methods. To address this challenge, we propose an efficient model-free feature selection framework via elastic expansion and compression of the features, namely EasyFS, to achieve better performance than state-of-the-art model-aware methods while sharing the characters of efficiency and flexibility with the existing model-free methods. In particular, EasyFS expands the feature space by using the random non-linear projection network to achieve the non-linear combinations of the original features, so as to model the interrelationships among the features and discover most correlated features. Meanwhile, a novel redundancy measurement based on the change of coding rate is proposed for efficient filtering of redundant features. Comprehensive experiments on 21 different datasets show that EasyFS outperforms state-of-the art methods up to 10.9\% in the regression tasks and 5.7\% in the classification tasks while saving more than 94\% of the time.

Related papers

• Causal Feature Selection Method for Contextual Multi-Armed Bandits in Recommender System [2.704084816922349]
  Feature selection methods for machine learning models fail short for contextual multi-armed bandits (MAB) use cases. In this paper, we introduce model-free feature selection methods designed for contexutal MAB problem. The results show this feature selection method effectively selects the important features that lead to higher contextual MAB reward.
  arXiv  Detail & Related papers  (2024-09-20T20:39:23Z)
• A Refreshed Similarity-based Upsampler for Direct High-Ratio Feature Upsampling [54.05517338122698]
  We propose an explicitly controllable query-key feature alignment from both semantic-aware and detail-aware perspectives. We also develop a fine-grained neighbor selection strategy on HR features, which is simple yet effective for alleviating mosaic artifacts. Our proposed ReSFU framework consistently achieves satisfactory performance on different segmentation applications.
  arXiv  Detail & Related papers  (2024-07-02T14:12:21Z)
• Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model [86.9619638550683]
  Vision-language foundation models have exhibited remarkable success across a multitude of downstream tasks due to their scalability on extensive image-text paired data. However, these models display significant limitations when applied to downstream tasks, such as fine-grained image classification, as a result of decision shortcuts''
  arXiv  Detail & Related papers  (2024-03-01T09:01:53Z)
• LoBaSS: Gauging Learnability in Supervised Fine-tuning Data [64.27898739929734]
  Supervised Fine-Tuning (SFT) serves as a crucial phase in aligning Large Language Models (LLMs) to specific task prerequisites. We introduce a new dimension in SFT data selection: learnability. We present the Loss Based SFT Data Selection (LoBaSS) method, utilizing data learnability as the principal criterion for the selection SFT data.
  arXiv  Detail & Related papers  (2023-10-16T07:26:24Z)
• FedSDG-FS: Efficient and Secure Feature Selection for Vertical Federated Learning [21.79965380400454]
  Vertical Learning (VFL) enables multiple data owners, each holding a different subset of features about largely overlapping sets of data sample(s) to jointly train a useful global model. Feature selection (FS) is important to VFL. It is still an open research problem as existing FS works designed for VFL either assumes prior knowledge on the number of noisy features or prior knowledge on the post-training threshold of useful features. We propose the Federated Dual-Gate based Feature Selection (FedSDG-FS) approach. It consists of a Gaussian dual-gate to efficiently approximate the probability of a feature being selected, with privacy
  arXiv  Detail & Related papers  (2023-02-21T03:09:45Z)
• Learning to Maximize Mutual Information for Dynamic Feature Selection [13.821253491768168]
  We consider the dynamic feature selection (DFS) problem where a model sequentially queries features based on the presently available information. We explore a simpler approach of greedily selecting features based on their conditional mutual information. The proposed method is shown to recover the greedy policy when trained to optimality, and it outperforms numerous existing feature selection methods in our experiments.
  arXiv  Detail & Related papers  (2023-01-02T08:31:56Z)
• Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning [126.84770886628833]
  Existing finetuning methods either tune all parameters of the pretrained model (full finetuning) or only tune the last linear layer (linear probing) We propose a new parameter-efficient finetuning method termed as SSF, representing that researchers only need to Scale and Shift the deep Features extracted by a pre-trained model to catch up with the performance full finetuning.
  arXiv  Detail & Related papers  (2022-10-17T08:14:49Z)
• Model-free feature selection to facilitate automatic discovery of divergent subgroups in tabular data [4.551615447454768]
  We propose a model-free and sparsity-based automatic feature selection (SAFS) framework to facilitate automatic discovery of divergent subgroups. We validated SAFS across two publicly available datasets (MIMIC-III and Allstate Claims) and compared it with six existing feature selection methods.
  arXiv  Detail & Related papers  (2022-03-08T20:42:56Z)
• Compactness Score: A Fast Filter Method for Unsupervised Feature Selection [66.84571085643928]
  We propose a fast unsupervised feature selection method, named as, Compactness Score (CSUFS) to select desired features. Our proposed algorithm seems to be more accurate and efficient compared with existing algorithms.
  arXiv  Detail & Related papers  (2022-01-31T13:01:37Z)
• Efficient Data-specific Model Search for Collaborative Filtering [56.60519991956558]
  Collaborative filtering (CF) is a fundamental approach for recommender systems. In this paper, motivated by the recent advances in automated machine learning (AutoML), we propose to design a data-specific CF model. Key here is a new framework that unifies state-of-the-art (SOTA) CF methods and splits them into disjoint stages of input encoding, embedding function, interaction and prediction function.
  arXiv  Detail & Related papers  (2021-06-14T14:30:32Z)
• Feature Selection Library (MATLAB Toolbox) [1.2058143465239939]
  The Feature Selection Library (FSLib) introduces a comprehensive suite of feature selection (FS) algorithms. FSLib addresses the curse of dimensionality, reduces computational load, and enhances model generalizability. FSLib contributes to data interpretability by revealing important features, aiding in pattern recognition and understanding.
  arXiv  Detail & Related papers  (2016-07-05T16:50:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.