Bagging Provides Assumption-free Stability

• URL: http://arxiv.org/abs/2301.12600v3
• Date: Thu, 25 Apr 2024 17:59:07 GMT
• Title: Bagging Provides Assumption-free Stability
• Authors: Jake A. Soloff, Rina Foygel Barber, Rebecca Willett,
• Abstract summary: Bagging is an important technique for stabilizing machine learning models. In this paper, we derive a finite-sample guarantee on the stability of bagging for any model.
• Score: 11.456416081243654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bagging is an important technique for stabilizing machine learning models. In this paper, we derive a finite-sample guarantee on the stability of bagging for any model. Our result places no assumptions on the distribution of the data, on the properties of the base algorithm, or on the dimensionality of the covariates. Our guarantee applies to many variants of bagging and is optimal up to a constant. Empirical results validate our findings, showing that bagging successfully stabilizes even highly unstable base algorithms.

Related papers

• Rigorous Probabilistic Guarantees for Robust Counterfactual Explanations [80.86128012438834]
  We show for the first time that computing the robustness of counterfactuals with respect to plausible model shifts is NP-complete. We propose a novel probabilistic approach which is able to provide tight estimates of robustness with strong guarantees.
  arXiv  Detail & Related papers  (2024-07-10T09:13:11Z)
• Stability Evaluation via Distributional Perturbation Analysis [28.379994938809133]
  We propose a stability evaluation criterion based on distributional perturbations. Our stability evaluation criterion can address both emphdata corruptions and emphsub-population shifts. Empirically, we validate the practical utility of our stability evaluation criterion across a host of real-world applications.
  arXiv  Detail & Related papers  (2024-05-06T06:47:14Z)
• Stable Update of Regression Trees [0.0]
  We focus on the stability of an inherently explainable machine learning method, namely regression trees. We propose a regularization method, where data points are weighted based on the uncertainty in the initial model. Results show that the proposed update method improves stability while achieving similar or better predictive performance.
  arXiv  Detail & Related papers  (2024-02-21T09:41:56Z)
• Stability Guarantees for Feature Attributions with Multiplicative Smoothing [11.675168649032875]
  We analyze stability as a property for reliable feature attribution methods. We develop a smoothing method called Multiplicative Smoothing (MuS) to achieve such a model. We evaluate MuS on vision and language models with various feature attribution methods, such as LIME and SHAP, and demonstrate that MuS endows feature attributions with non-trivial stability guarantees.
  arXiv  Detail & Related papers  (2023-07-12T04:19:47Z)
• Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval [51.83967175585896]
  UAL aims at providing reliability-aware predictions by considering data uncertainty and model uncertainty simultaneously. Data uncertainty captures the noise" inherent in the sample, while model uncertainty depicts the model's confidence in the sample's prediction.
  arXiv  Detail & Related papers  (2022-10-24T17:53:20Z)
• Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Trees [57.67528738886731]
  We study the numerical stability of scalable sparse approximations based on inducing points. For low-dimensional tasks such as geospatial modeling, we propose an automated method for computing inducing points satisfying these conditions.
  arXiv  Detail & Related papers  (2022-10-14T15:20:17Z)
• Black box tests for algorithmic stability [10.495496415022064]
  Knowing an algorithm's stability properties is often useful for many downstream applications. Many modern algorithms currently used in practice are too complex for a theoretical analysis of their stability properties.
  arXiv  Detail & Related papers  (2021-11-30T16:36:58Z)
• Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
  Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design. We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
  arXiv  Detail & Related papers  (2021-05-17T08:36:18Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent [55.85456985750134]
  We introduce a new stability measure called on-average model stability, for which we develop novel bounds controlled by the risks of SGD iterates. This yields generalization bounds depending on the behavior of the best model, and leads to the first-ever-known fast bounds in the low-noise setting. To our best knowledge, this gives the firstever-known stability and generalization for SGD with even non-differentiable loss functions.
  arXiv  Detail & Related papers  (2020-06-15T06:30:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.