Top-$k$ Feature Importance Ranking

• URL: http://arxiv.org/abs/2509.15420v1
• Date: Thu, 18 Sep 2025 20:58:30 GMT
• Title: Top-$k$ Feature Importance Ranking
• Authors: Yuxi Chen, Tiffany Tang, Genevera Allen,
• Abstract summary: RAMPART is a framework that utilizes any existing feature importance measure in a novel algorithm specifically tailored for ranking the top-$k$ features.<n>Our approach combines an adaptive halving sequential strategy that progressively focuses computational resources on promising features.<n>We provide theoretical guarantees showing that RAMPART achieves the correct top-$k$ ranking with high probability under mild conditions.
• Score: 0.2906550609733701
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate ranking of important features is a fundamental challenge in interpretable machine learning with critical applications in scientific discovery and decision-making. Unlike feature selection and feature importance, the specific problem of ranking important features has received considerably less attention. We introduce RAMPART (Ranked Attributions with MiniPatches And Recursive Trimming), a framework that utilizes any existing feature importance measure in a novel algorithm specifically tailored for ranking the top-$k$ features. Our approach combines an adaptive sequential halving strategy that progressively focuses computational resources on promising features with an efficient ensembling technique using both observation and feature subsampling. Unlike existing methods that convert importance scores to ranks as post-processing, our framework explicitly optimizes for ranking accuracy. We provide theoretical guarantees showing that RAMPART achieves the correct top-$k$ ranking with high probability under mild conditions, and demonstrate through extensive simulation studies that RAMPART consistently outperforms popular feature importance methods, concluding with a high-dimensional genomics case study.

Related papers

• Explicit and Non-asymptotic Query Complexities of Rank-Based Zeroth-order Algorithm on Stochastic Smooth Functions [17.238068736229014]
  Zeroth-order (ZO) optimization with ordinal feedback has emerged as a fundamental problem in modern machine learning systems.<n>We propose a simple and computationally efficient rank-based ZO algorithm.
  arXiv  Detail & Related papers  (2025-12-22T07:18:57Z)
• Large-scale Multi-objective Feature Selection: A Multi-phase Search Space Shrinking Approach [0.27624021966289597]
  Feature selection is a crucial step in machine learning, especially for high-dimensional datasets. This paper proposes a novel large-scale multi-objective evolutionary algorithm based on the search space shrinking, termed LMSSS. The effectiveness of the proposed algorithm is demonstrated through comprehensive experiments on 15 large-scale datasets.
  arXiv  Detail & Related papers  (2024-10-13T23:06:10Z)
• Statistical Significance of Feature Importance Rankings [3.8642937395065124]
  We devise techniques that ensure the most important features are correct with high-probability guarantees.<n>These assess the set of $K$ top-ranked features, as well as the order of its elements.<n>We then introduce two efficient sampling algorithms that identify the $K$ most important features, perhaps in order, with probability exceeding $1-alpha$.
  arXiv  Detail & Related papers  (2024-01-28T23:14:51Z)
• Confident Feature Ranking [2.0564549686015594]
  We present a framework for quantifying the uncertainty in global importance values. We propose a novel method for the post-hoc interpretation of feature importance values.
  arXiv  Detail & Related papers  (2023-07-28T07:23:01Z)
• Efficient Model-Free Exploration in Low-Rank MDPs [76.87340323826945]
  Low-Rank Markov Decision Processes offer a simple, yet expressive framework for RL with function approximation. Existing algorithms are either (1) computationally intractable, or (2) reliant upon restrictive statistical assumptions. We propose the first provably sample-efficient algorithm for exploration in Low-Rank MDPs.
  arXiv  Detail & Related papers  (2023-07-08T15:41:48Z)
• MARS: Meta-Learning as Score Matching in the Function Space [79.73213540203389]
  We present a novel approach to extracting inductive biases from a set of related datasets. We use functional Bayesian neural network inference, which views the prior as a process and performs inference in the function space. Our approach can seamlessly acquire and represent complex prior knowledge by metalearning the score function of the data-generating process.
  arXiv  Detail & Related papers  (2022-10-24T15:14:26Z)
• Deep Feature Selection Using a Novel Complementary Feature Mask [5.904240881373805]
  We deal with feature selection by exploiting the features with less importance scores. We propose a feature selection framework based on a novel complementary feature mask. Our method is generic and can be easily integrated into existing deep-learning-based feature selection approaches.
  arXiv  Detail & Related papers  (2022-09-25T18:03:30Z)
• Compactness Score: A Fast Filter Method for Unsupervised Feature Selection [66.84571085643928]
  We propose a fast unsupervised feature selection method, named as, Compactness Score (CSUFS) to select desired features. Our proposed algorithm seems to be more accurate and efficient compared with existing algorithms.
  arXiv  Detail & Related papers  (2022-01-31T13:01:37Z)
• An Efficient and Accurate Rough Set for Feature Selection, Classification and Knowledge Representation [89.5951484413208]
  This paper present a strong data mining method based on rough set, which can realize feature selection, classification and knowledge representation at the same time. We first find the ineffectiveness of rough set because of overfitting, especially in processing noise attribute, and propose a robust measurement for an attribute, called relative importance. Experimental results on public benchmark data sets show that the proposed framework achieves higher accurcy than seven popular or the state-of-the-art feature selection methods.
  arXiv  Detail & Related papers  (2021-12-29T12:45:49Z)
• Loss Function Search for Face Recognition [75.79325080027908]
  We develop a reward-guided search method to automatically obtain the best candidate. Experimental results on a variety of face recognition benchmarks have demonstrated the effectiveness of our method.
  arXiv  Detail & Related papers  (2020-07-10T03:40:10Z)
• Learning to Learn Kernels with Variational Random Features [118.09565227041844]
  We introduce kernels with random Fourier features in the meta-learning framework to leverage their strong few-shot learning ability. We formulate the optimization of MetaVRF as a variational inference problem. We show that MetaVRF delivers much better, or at least competitive, performance compared to existing meta-learning alternatives.
  arXiv  Detail & Related papers  (2020-06-11T18:05:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.