Distributionally Robust Safe Screening

• URL: http://arxiv.org/abs/2404.16328v1
• Date: Thu, 25 Apr 2024 04:29:25 GMT
• Title: Distributionally Robust Safe Screening
• Authors: Hiroyuki Hanada, Satoshi Akahane, Tatsuya Aoyama, Tomonari Tanaka, Yoshito Okura, Yu Inatsu, Noriaki Hashimoto, Taro Murayama, Lee Hanju, Shinya Kojima, Ichiro Takeuchi,
• Abstract summary: We propose a Distributionally Robust Safe Screening (DRSS) method for identifying unnecessary samples and features. We provide a theoretical guarantee of the DRSS method and validate its performance through numerical experiments on both synthetic and real-world datasets.
• Score: 14.973247943788234
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this study, we propose a method Distributionally Robust Safe Screening (DRSS), for identifying unnecessary samples and features within a DR covariate shift setting. This method effectively combines DR learning, a paradigm aimed at enhancing model robustness against variations in data distribution, with safe screening (SS), a sparse optimization technique designed to identify irrelevant samples and features prior to model training. The core concept of the DRSS method involves reformulating the DR covariate-shift problem as a weighted empirical risk minimization problem, where the weights are subject to uncertainty within a predetermined range. By extending the SS technique to accommodate this weight uncertainty, the DRSS method is capable of reliably identifying unnecessary samples and features under any future distribution within a specified range. We provide a theoretical guarantee of the DRSS method and validate its performance through numerical experiments on both synthetic and real-world datasets.

Related papers

• Distributionally Robust Safe Sample Elimination under Covariate Shift [16.85444622474742]
  We consider a machine learning setup where one training dataset is used to train multiple models across slightly different data distributions. We propose the DRSSS method, which combines distributionally robust (DR) optimization and safe sample screening (SSS) The key benefit of this method is that models trained on the reduced dataset will perform the same as those trained on the full dataset for all possible different environments.
  arXiv  Detail & Related papers  (2024-06-10T01:46:42Z)
• The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness [50.52507648690234]
  Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model. We introduce three robustness indicators and conduct experiments across diverse robust datasets. Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-01-25T09:18:51Z)
• [Re] Double Sampling Randomized Smoothing [2.6763498831034043]
  This paper is a contribution to the challenge in the field of machine learning, specifically addressing the issue of certifying the robustness of neural networks (NNs) against adversarial perturbations. The proposed Double Sampling Randomized Smoothing (DSRS) framework overcomes the limitations of existing methods by using an additional smoothing distribution to improve the robustness certification.
  arXiv  Detail & Related papers  (2023-06-27T05:46:18Z)
• SafeDiffuser: Safe Planning with Diffusion Probabilistic Models [97.80042457099718]
  Diffusion model-based approaches have shown promise in data-driven planning, but there are no safety guarantees. We propose a new method, called SafeDiffuser, to ensure diffusion probabilistic models satisfy specifications. We test our method on a series of safe planning tasks, including maze path generation, legged robot locomotion, and 3D space manipulation.
  arXiv  Detail & Related papers  (2023-05-31T19:38:12Z)
• Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty [68.00748155945047]
  Capturing uncertainty in models of complex dynamical systems is crucial to designing safe controllers. Several approaches use formal abstractions to synthesize policies that satisfy temporal specifications related to safety and reachability. Our contribution is a novel abstraction-based controller method for continuous-state models with noise, uncertain parameters, and external disturbances.
  arXiv  Detail & Related papers  (2022-10-12T07:57:03Z)
• Adaptive novelty detection with false discovery rate guarantee [1.8249324194382757]
  We propose a flexible method to control the false discovery rate (FDR) on detected novelties in finite samples. Inspired by the multiple testing literature, we propose variants of AdaDetect that are adaptive to the proportion of nulls. The methods are illustrated on synthetic datasets and real-world datasets, including an application in astrophysics.
  arXiv  Detail & Related papers  (2022-08-13T17:14:55Z)
• Distributed Dynamic Safe Screening Algorithms for Sparse Regularization [73.85961005970222]
  We propose a new distributed dynamic safe screening (DDSS) method for sparsity regularized models and apply it on shared-memory and distributed-memory architecture respectively. We prove that the proposed method achieves the linear convergence rate with lower overall complexity and can eliminate almost all the inactive features in a finite number of iterations almost surely.
  arXiv  Detail & Related papers  (2022-04-23T02:45:55Z)
• Residuals-based distributionally robust optimization with covariate information [0.0]
  We consider data-driven approaches that integrate a machine learning prediction model within distributionally robust optimization (DRO) Our framework is flexible in the sense that it can accommodate a variety of learning setups and DRO ambiguity sets.
  arXiv  Detail & Related papers  (2020-12-02T11:21:34Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network [66.26231423824089]
  We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models.
  arXiv  Detail & Related papers  (2020-03-04T12:27:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.