Dynamic Feature Selection from Variable Feature Sets Using Features of Features

• URL: http://arxiv.org/abs/2503.09181v1
• Date: Wed, 12 Mar 2025 09:13:21 GMT
• Title: Dynamic Feature Selection from Variable Feature Sets Using Features of Features
• Abstract summary: We propose a new problem setting for Dynamic Feature Selection (DFS) with variable feature sets.<n>We show that the proposed method effectively selects features based on the prior information, even when the set of measurable features changes from instance to instance.
• Score: 24.668942936732083
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning models usually assume that a set of feature values used to obtain an output is fixed in advance. However, in many real-world problems, a cost is associated with measuring these features. To address the issue of reducing measurement costs, various methods have been proposed to dynamically select which features to measure, but existing methods assume that the set of measurable features remains constant, which makes them unsuitable for cases where the set of measurable features varies from instance to instance. To overcome this limitation, we define a new problem setting for Dynamic Feature Selection (DFS) with variable feature sets and propose a deep learning method that utilizes prior information about each feature, referred to as ''features of features''. Experimental results on several datasets demonstrate that the proposed method effectively selects features based on the prior information, even when the set of measurable features changes from instance to instance.

Related papers

• Hide&Seek: Learning to Explain in an End-to-End Differentiable Network [3.2880869992413246]
  We present Hide&Seek, an end-to-end differentiable model for instance-wise feature selection.<n>We jointly learn feature selection and prediction under a single objective without information leakage.<n>We achieve this by reformulating feature removal as a differentiable operation where instead of discretely removing features, we replace a proportion of each feature.
  arXiv  Detail & Related papers  (2026-08-17T15:10:05Z)
• Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects [58.98080129016082]
  Feature-Effect Geometry Analysis (FEGA) is an unsupervised framework that removes the same active SAE feature across contexts.<n>We show that a feature can be interpretable and causally relevant without providing a stable direction for steering.
  arXiv  Detail & Related papers  (2026-07-27T16:45:08Z)
• Stochastic Encodings for Active Feature Acquisition [100.47043816019888]
  Active Feature Acquisition is an instance-wise, sequential decision making problem.<n>The aim is to dynamically select which feature to measure based on current observations, independently for each test instance.<n>Common approaches either use Reinforcement Learning, which experiences training difficulties, or greedily maximize the conditional mutual information of the label and unobserved features, which makes myopic.<n>We introduce a latent variable model, trained in a supervised manner. Acquisitions are made by reasoning about the features across many possible unobserved realizations in a latent space.
  arXiv  Detail & Related papers  (2025-08-03T23:48:46Z)
• Permutation-based multi-objective evolutionary feature selection for high-dimensional data [43.18726655647964]
  We propose a novel feature selection method for high-dimensional data, based on the well-known permutation feature importance approach.<n>The proposed method employs a multi-objective evolutionary algorithm to search for candidate feature subsets.<n>The effectiveness of our method has been validated on a set of 24 publicly available high-dimensional datasets.
  arXiv  Detail & Related papers  (2025-01-24T08:11:28Z)
• Feature Selection as Deep Sequential Generative Learning [50.00973409680637]
  We develop a deep variational transformer model over a joint of sequential reconstruction, variational, and performance evaluator losses. Our model can distill feature selection knowledge and learn a continuous embedding space to map feature selection decision sequences into embedding vectors associated with utility scores.
  arXiv  Detail & Related papers  (2024-03-06T16:31:56Z)
• A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning [131.2910403490434]
  Data scientists typically collect as many features as possible into their datasets, and even engineer new features from existing ones. Existing benchmarks for tabular feature selection consider classical downstream models, toy synthetic datasets, or do not evaluate feature selectors on the basis of downstream performance. We construct a challenging feature selection benchmark evaluated on downstream neural networks including transformers. We also propose an input-gradient-based analogue of Lasso for neural networks that outperforms classical feature selection methods on challenging problems.
  arXiv  Detail & Related papers  (2023-11-10T05:26:10Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• Finding Optimal Diverse Feature Sets with Alternative Feature Selection [0.0]
  We introduce alternative feature selection and formalize it as an optimization problem.<n>In particular, we define alternatives via constraints and enable users to control the number and dissimilarity of alternatives.<n>We show that a constant-factor approximation exists under certain conditions and propose corresponding search methods.
  arXiv  Detail & Related papers  (2023-07-21T14:23:41Z)
• Feature Selection with Distance Correlation [0.0]
  We develop a new feature selection method based on Distance Correlation (DisCo) Using our method to select features from a set of over 7,000 energy flows, we show that we can match the performance of much deeper architectures.
  arXiv  Detail & Related papers  (2022-11-30T19:00:04Z)
• VFDS: Variational Foresight Dynamic Selection in Bayesian Neural Networks for Efficient Human Activity Recognition [81.29900407096977]
  Variational Foresight Dynamic Selection (VFDS) learns a policy that selects the next feature subset to observe. We apply VFDS on the Human Activity Recognition (HAR) task where the performance-cost trade-off is critical in its practice.
  arXiv  Detail & Related papers  (2022-03-31T22:52:43Z)
• Few-shot Learning for Unsupervised Feature Selection [59.75321498170363]
  We propose a few-shot learning method for unsupervised feature selection. The proposed method can select a subset of relevant features in a target task given a few unlabeled target instances. We experimentally demonstrate that the proposed method outperforms existing feature selection methods.
  arXiv  Detail & Related papers  (2021-07-02T03:52:51Z)
• Dynamic Instance-Wise Classification in Correlated Feature Spaces [15.351282873821935]
  In a typical machine learning setting, the predictions on all test instances are based on a common subset of features discovered during model training. A new method is proposed that sequentially selects the best feature to evaluate for each test instance individually, and stops the selection process to make a prediction once it determines that no further improvement can be achieved with respect to classification accuracy. The effectiveness, generalizability, and scalability of the proposed method is illustrated on a variety of real-world datasets from diverse application domains.
  arXiv  Detail & Related papers  (2021-06-08T20:20:36Z)
• Feature Selection Using Reinforcement Learning [0.0]
  The space of variables or features that can be used to characterize a particular predictor of interest continues to grow exponentially. Identifying the most characterizing features that minimizes the variance without jeopardizing the bias of our models is critical to successfully training a machine learning model.
  arXiv  Detail & Related papers  (2021-01-23T09:24:37Z)
• Feature Selection for Huge Data via Minipatch Learning [0.0]
  We propose Stable Minipatch Selection (STAMPS) and Adaptive STAMPS. STAMPS are meta-algorithms that build ensembles of selection events of base feature selectors trained on tiny, (ly-adaptive) random subsets of both the observations and features of the data. Our approaches are general and can be employed with a variety of existing feature selection strategies and machine learning techniques.
  arXiv  Detail & Related papers  (2020-10-16T17:41:08Z)
• The role of feature space in atomistic learning [62.997667081978825]
  Physically-inspired descriptors play a key role in the application of machine-learning techniques to atomistic simulations. We introduce a framework to compare different sets of descriptors, and different ways of transforming them by means of metrics and kernels. We compare representations built in terms of n-body correlations of the atom density, quantitatively assessing the information loss associated with the use of low-order features.
  arXiv  Detail & Related papers  (2020-09-06T14:12:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.