Sparse minimum Redundancy Maximum Relevance for feature selection

• URL: http://arxiv.org/abs/2508.18901v1
• Date: Tue, 26 Aug 2025 10:18:23 GMT
• Title: Sparse minimum Redundancy Maximum Relevance for feature selection
• Authors: Peter Naylor, Benjamin Poignard, Héctor Climente-González, Makoto Yamada,
• Abstract summary: Inactive features are identified via a penalized feature screening procedure.<n>Our method performs comparably to HLAS-SO but is more conservative in the number of selected features.<n>It only requires setting an FDR threshold, rather than the effectiveness of the method is illustrated through simulations.
• Score: 14.65479063982159
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a feature screening method that integrates both feature-feature and feature-target relationships. Inactive features are identified via a penalized minimum Redundancy Maximum Relevance (mRMR) procedure, which is the continuous version of the classic mRMR penalized by a non-convex regularizer, and where the parameters estimated as zero coefficients represent the set of inactive features. We establish the conditions under which zero coefficients are correctly identified to guarantee accurate recovery of inactive features. We introduce a multi-stage procedure based on the knockoff filter enabling the penalized mRMR to discard inactive features while controlling the false discovery rate (FDR). Our method performs comparably to HSIC-LASSO but is more conservative in the number of selected features. It only requires setting an FDR threshold, rather than specifying the number of features to retain. The effectiveness of the method is illustrated through simulations and real-world datasets. The code to reproduce this work is available on the following GitHub: https://github.com/PeterJackNaylor/SmRMR.

Related papers

• QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation [52.024845354511555]
  We propose QR-LoRA, a novel fine-tuning framework leveraging QR decomposition for structured parameter updates.<n>Our key insight is that the Q matrix naturally minimizes interference between different visual features.<n>Experiments demonstrate that QR-LoRA achieves superior disentanglement in content-style fusion tasks.
  arXiv  Detail & Related papers  (2025-07-07T01:31:01Z)
• HCVR: A Hybrid Approach with Correlation-aware Voting Rules for Feature Selection [0.0]
  HCVR (Hybrid approach with Correlation-aware Voting Rules) is a lightweight rule-based feature selection method.<n>It combines -to-one correlations to eliminate redundant features and relevant ones.<n>Results show improvement as compared to traditional non-iterative (CFS, mRMR and MI) and iterative (RFE, SFS and Genetic) techniques.
  arXiv  Detail & Related papers  (2025-07-02T18:20:56Z)
• Reinforced Model Merging [53.84354455400038]
  We present an innovative framework termed Reinforced Model Merging (RMM), which encompasses an environment and agent tailored for merging tasks.<n>By utilizing data subsets during the evaluation process, we addressed the bottleneck in the reward feedback phase, thereby accelerating RMM by up to 100 times.
  arXiv  Detail & Related papers  (2025-03-27T08:52:41Z)
• Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model [66.91323540178739]
  Sequential recommendation (SR) aims to predict items that users may be interested in based on their historical behavior. We revisit SR from a novel information-theoretic perspective and find that sequential modeling methods fail to adequately capture randomness and unpredictability of user behavior. Inspired by fuzzy information processing theory, this paper introduces the fuzzy sets of interaction sequences to overcome the limitations and better capture the evolution of users' real interests.
  arXiv  Detail & Related papers  (2024-10-31T14:52:01Z)
• Is the MMI Criterion Necessary for Interpretability? Degenerating Non-causal Features to Plain Noise for Self-Rationalization [17.26418974819275]
  This paper develops a new criterion that treats spurious features as plain noise. Experiments show that our MRD criterion improves rationale quality (measured by the overlap with human-annotated rationales) by up to $10.4%$ as compared to several recent competitive MMI variants.
  arXiv  Detail & Related papers  (2024-10-08T13:04:02Z)
• Conformal Recursive Feature Elimination [0.0]
  Conformal Prediction (CP) allows for the determination of valid and accurate confidence levels associated with individual predictions. We introduce a new feature selection method that takes advantage of the CP framework.
  arXiv  Detail & Related papers  (2024-05-29T18:10:36Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• MVMR-FS : Non-parametric feature selection algorithm based on Maximum inter-class Variation and Minimum Redundancy [1.2522889958051284]
  We propose a non-parametric feature selection algorithm based on maximum inter-class variation and minimum redundancy. Compared with ten state-of-the-art methods, MVMR-FS achieves the highest average accuracy and improves the accuracy by 5% to 11%.
  arXiv  Detail & Related papers  (2023-07-27T06:33:17Z)
• Optimal Activation Functions for the Random Features Regression Model [7.381113319198103]
  We identify in closed-form the family of Activation Functions (AFs) that minimize a combination of the test error and sensitivity of the Random Features Regression model. We show how using optimal AFs impacts well-established properties of the RFR model.
  arXiv  Detail & Related papers  (2022-05-31T16:41:42Z)
• Error-based Knockoffs Inference for Controlled Feature Selection [49.99321384855201]
  We propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together. The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees.
  arXiv  Detail & Related papers  (2022-03-09T01:55:59Z)
• Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficient [62.24615324523435]
  This paper provides a statistical analysis of high-dimensional batch Reinforcement Learning (RL) using sparse linear function approximation. When there is a large number of candidate features, our result sheds light on the fact that sparsity-aware methods can make batch RL more sample efficient.
  arXiv  Detail & Related papers  (2020-11-08T16:48:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.