Fast Estimation Method for the Stability of Ensemble Feature Selectors

• URL: http://arxiv.org/abs/2108.01485v1
• Date: Tue, 3 Aug 2021 13:22:18 GMT
• Title: Fast Estimation Method for the Stability of Ensemble Feature Selectors
• Authors: Rina Onda, Zhengyan Gao, Masaaki Kotera, Kenta Oono
• Abstract summary: It is preferred that feature selectors be textitstable for better interpretabity and robust prediction. We propose a simulator of a feature selector, and apply it to a fast estimation of the stability of ensemble feature selectors.
• Score: 8.984888893275714
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It is preferred that feature selectors be \textit{stable} for better interpretabity and robust prediction. Ensembling is known to be effective for improving the stability of feature selectors. Since ensembling is time-consuming, it is desirable to reduce the computational cost to estimate the stability of the ensemble feature selectors. We propose a simulator of a feature selector, and apply it to a fast estimation of the stability of ensemble feature selectors. To the best of our knowledge, this is the first study that estimates the stability of ensemble feature selectors and reduces the computation time theoretically and empirically.

Related papers

• Integrated path stability selection [5.263910852465186]
  We introduce a novel approach to stability selection based on integrating stability paths rather than maximizing over them. This yields upper bounds on E(FP) that are orders of magnitude stronger than previous bounds, leading to significantly more true positives in practice for the same target E(FP) We demonstrate the method on simulations and real data from prostate and colon cancer studies.
  arXiv  Detail & Related papers  (2024-03-23T15:55:52Z)
• Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model [86.9619638550683]
  Vision-language foundation models have exhibited remarkable success across a multitude of downstream tasks due to their scalability on extensive image-text paired data. However, these models display significant limitations when applied to downstream tasks, such as fine-grained image classification, as a result of decision shortcuts''
  arXiv  Detail & Related papers  (2024-03-01T09:01:53Z)
• Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Trees [57.67528738886731]
  We study the numerical stability of scalable sparse approximations based on inducing points. For low-dimensional tasks such as geospatial modeling, we propose an automated method for computing inducing points satisfying these conditions.
  arXiv  Detail & Related papers  (2022-10-14T15:20:17Z)
• Loss-guided Stability Selection [0.0]
  It is well-known that model selection procedures like the Lasso or Boosting tend to overfit on real data. Standard Stability Selection is based on a global criterion, namely the per-family error rate. We propose a Stability Selection variant which respects the chosen loss function via an additional validation step.
  arXiv  Detail & Related papers  (2022-02-10T11:20:25Z)
• Cluster Stability Selection [2.3986080077861787]
  Stability selection makes any feature selection method more stable by returning only those features that are consistently selected across many subsamples. We introduce cluster stability selection, which exploits the practitioner's knowledge that highly correlated clusters exist in the data. In summary, cluster stability selection enjoys the best of both worlds, yielding a sparse selected set that is both stable and has good predictive performance.
  arXiv  Detail & Related papers  (2022-01-03T06:28:17Z)
• Optimal Rates for Random Order Online Optimization [60.011653053877126]
  We study the citetgarber 2020online, where the loss functions may be chosen by an adversary, but are then presented online in a uniformly random order. We show that citetgarber 2020online algorithms achieve the optimal bounds and significantly improve their stability.
  arXiv  Detail & Related papers  (2021-06-29T09:48:46Z)
• Employing an Adjusted Stability Measure for Multi-Criteria Model Fitting on Data Sets with Similar Features [0.1127980896956825]
  We show that our approach achieves the same or better predictive performance compared to the two established approaches. Our approach succeeds at selecting the relevant features while avoiding irrelevant or redundant features. For data sets with many similar features, the feature selection stability must be evaluated with an adjusted stability measure.
  arXiv  Detail & Related papers  (2021-06-15T12:48:07Z)
• Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
  Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design. We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
  arXiv  Detail & Related papers  (2021-05-17T08:36:18Z)
• Learning Stabilizing Controllers for Unstable Linear Quadratic Regulators from a Single Trajectory [85.29718245299341]
  We study linear controllers under quadratic costs model also known as linear quadratic regulators (LQR) We present two different semi-definite programs (SDP) which results in a controller that stabilizes all systems within an ellipsoid uncertainty set. We propose an efficient data dependent algorithm -- textsceXploration -- that with high probability quickly identifies a stabilizing controller.
  arXiv  Detail & Related papers  (2020-06-19T08:58:57Z)
• Leveraging Model Inherent Variable Importance for Stable Online Feature Selection [16.396739487911056]
  We introduce FIRES, a novel framework for online feature selection. Our framework is generic in that it leaves the choice of the underlying model to the user. Experiments show that the proposed framework is clearly superior in terms of feature selection stability.
  arXiv  Detail & Related papers  (2020-06-18T10:01:18Z)
• Distributionally Robust Bayesian Optimization [121.71766171427433]
  We present a novel distributionally robust Bayesian optimization algorithm (DRBO) for zeroth-order, noisy optimization. Our algorithm provably obtains sub-linear robust regret in various settings. We demonstrate the robust performance of our method on both synthetic and real-world benchmarks.
  arXiv  Detail & Related papers  (2020-02-20T22:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.