Robustness of Machine Learning Models Beyond Adversarial Attacks
- URL: http://arxiv.org/abs/2204.10046v1
- Date: Thu, 21 Apr 2022 12:09:49 GMT
- Title: Robustness of Machine Learning Models Beyond Adversarial Attacks
- Authors: Sebastian Scher and Andreas Tr\"ugler
- Abstract summary: We show that the widely used concept of adversarial robustness and closely related metrics are not necessarily valid metrics for determining the robustness of ML models.
We propose a flexible approach that models possible perturbations in input data individually for each application.
This is then combined with a probabilistic approach that computes the likelihood that a real-world perturbation will change a prediction.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Correctly quantifying the robustness of machine learning models is a central
aspect in judging their suitability for specific tasks, and thus, ultimately,
for generating trust in the models. We show that the widely used concept of
adversarial robustness and closely related metrics based on counterfactuals are
not necessarily valid metrics for determining the robustness of ML models
against perturbations that occur "naturally", outside specific adversarial
attack scenarios. Additionally, we argue that generic robustness metrics in
principle are insufficient for determining real-world-robustness. Instead we
propose a flexible approach that models possible perturbations in input data
individually for each application. This is then combined with a probabilistic
approach that computes the likelihood that a real-world perturbation will
change a prediction, thus giving quantitative information of the robustness of
the trained machine learning model. The method does not require access to the
internals of the classifier and thus in principle works for any black-box
model. It is, however, based on Monte-Carlo sampling and thus only suited for
input spaces with small dimensions. We illustrate our approach on two dataset,
as well as on analytically solvable cases. Finally, we discuss ideas on how
real-world robustness could be computed or estimated in high-dimensional input
spaces.
Related papers
- Rigorous Probabilistic Guarantees for Robust Counterfactual Explanations [80.86128012438834]
We show for the first time that computing the robustness of counterfactuals with respect to plausible model shifts is NP-complete.
We propose a novel probabilistic approach which is able to provide tight estimates of robustness with strong guarantees.
arXiv Detail & Related papers (2024-07-10T09:13:11Z) - LoRA-Ensemble: Efficient Uncertainty Modelling for Self-attention Networks [52.46420522934253]
We introduce LoRA-Ensemble, a parameter-efficient deep ensemble method for self-attention networks.
By employing a single pre-trained self-attention network with weights shared across all members, we train member-specific low-rank matrices for the attention projections.
Our method exhibits superior calibration compared to explicit ensembles and achieves similar or better accuracy across various prediction tasks and datasets.
arXiv Detail & Related papers (2024-05-23T11:10:32Z) - Variational Shapley Network: A Probabilistic Approach to Self-Explaining
Shapley values with Uncertainty Quantification [2.6699011287124366]
Shapley values have emerged as a foundational tool in machine learning (ML) for elucidating model decision-making processes.
We introduce a novel, self-explaining method that simplifies the computation of Shapley values significantly, requiring only a single forward pass.
arXiv Detail & Related papers (2024-02-06T18:09:05Z) - Characterizing Data Point Vulnerability via Average-Case Robustness [29.881355412540557]
adversarial robustness is a standard framework, which views robustness of predictions through a binary lens.
We consider a complementary framework for robustness, called average-case robustness, which measures the fraction of points in a local region.
We show empirically that our estimators are accurate and efficient for standard deep learning models.
arXiv Detail & Related papers (2023-07-26T01:10:29Z) - Quantile-constrained Wasserstein projections for robust interpretability
of numerical and machine learning models [18.771531343438227]
The study of black-box models is often based on sensitivity analysis involving a probabilistic structure imposed on the inputs.
Our work aim at unifying the UQ and ML interpretability approaches, by providing relevant and easy-to-use tools for both paradigms.
arXiv Detail & Related papers (2022-09-23T11:58:03Z) - CC-Cert: A Probabilistic Approach to Certify General Robustness of
Neural Networks [58.29502185344086]
In safety-critical machine learning applications, it is crucial to defend models against adversarial attacks.
It is important to provide provable guarantees for deep learning models against semantically meaningful input transformations.
We propose a new universal probabilistic certification approach based on Chernoff-Cramer bounds.
arXiv Detail & Related papers (2021-09-22T12:46:04Z) - Trust but Verify: Assigning Prediction Credibility by Counterfactual
Constrained Learning [123.3472310767721]
Prediction credibility measures are fundamental in statistics and machine learning.
These measures should account for the wide variety of models used in practice.
The framework developed in this work expresses the credibility as a risk-fit trade-off.
arXiv Detail & Related papers (2020-11-24T19:52:38Z) - Modal Uncertainty Estimation via Discrete Latent Representation [4.246061945756033]
We introduce a deep learning framework that learns the one-to-many mappings between the inputs and outputs, together with faithful uncertainty measures.
Our framework demonstrates significantly more accurate uncertainty estimation than the current state-of-the-art methods.
arXiv Detail & Related papers (2020-07-25T05:29:34Z) - Machine learning for causal inference: on the use of cross-fit
estimators [77.34726150561087]
Doubly-robust cross-fit estimators have been proposed to yield better statistical properties.
We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE)
When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
arXiv Detail & Related papers (2020-04-21T23:09:55Z) - Meta-Learned Confidence for Few-shot Learning [60.6086305523402]
A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples.
We propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries.
We validate our few-shot learning model with meta-learned confidence on four benchmark datasets.
arXiv Detail & Related papers (2020-02-27T10:22:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.