Provable Robust Saliency-based Explanations
- URL: http://arxiv.org/abs/2212.14106v3
- Date: Sat, 8 Jul 2023 17:57:36 GMT
- Title: Provable Robust Saliency-based Explanations
- Authors: Chao Chen, Chenghua Guo, Guixiang Ma, Ming Zeng, Xi Zhang, Sihong Xie
- Abstract summary: We show that R2ET attains higher explanation robustness under stealthy attacks while retaining model accuracy.
Experiments with a wide spectrum of network architectures and data modalities demonstrate that R2ET attains higher explanation robustness under stealthy attacks while retaining model accuracy.
- Score: 16.217374556142484
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Robust explanations of machine learning models are critical to establishing
human trust in the models. The top-$k$ intersection is widely used to evaluate
the robustness of explanations. However, most existing attacking and defense
strategies are based on $\ell_p$ norms, thus creating a mismatch between the
evaluation and optimization objectives. To this end, we define explanation
thickness for measuring top-$k$ salient features ranking stability, and design
the \textit{R2ET} algorithm based on a novel tractable surrogate to maximize
the thickness and stabilize the top salient features efficiently.
Theoretically, we prove a connection between R2ET and adversarial training;
using a novel multi-objective optimization formulation and a generalization
error bound, we further prove that the surrogate objective can improve both the
numerical and statistical stability of the explanations. Experiments with a
wide spectrum of network architectures and data modalities demonstrate that
R2ET attains higher explanation robustness under stealthy attacks while
retaining model accuracy.
Related papers
- RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects [12.5122702720856]
We propose Robust Fine-Tuning (RbFT) to enhance the resilience of large language models against retrieval defects.
Experimental results demonstrate that RbFT significantly improves the robustness of RAG systems across diverse retrieval conditions.
arXiv Detail & Related papers (2025-01-30T14:15:09Z) - Stability Evaluation via Distributional Perturbation Analysis [28.379994938809133]
We propose a stability evaluation criterion based on distributional perturbations.
Our stability evaluation criterion can address both emphdata corruptions and emphsub-population shifts.
Empirically, we validate the practical utility of our stability evaluation criterion across a host of real-world applications.
arXiv Detail & Related papers (2024-05-06T06:47:14Z) - The Risk of Federated Learning to Skew Fine-Tuning Features and
Underperform Out-of-Distribution Robustness [50.52507648690234]
Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model.
We introduce three robustness indicators and conduct experiments across diverse robust datasets.
Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
arXiv Detail & Related papers (2024-01-25T09:18:51Z) - Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
We propose a novel doubly-robust instance reweighted adversarial framework.
Our importance weights are obtained by optimizing the KL-divergence regularized loss function.
Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
arXiv Detail & Related papers (2023-08-01T06:16:18Z) - A Stability Analysis of Fine-Tuning a Pre-Trained Model [46.6761331971071]
Fine-tuning a pre-trained model is one of the most promising paradigms in recent NLP research.
Fine-tuning suffers from the instability problem, i.e., tuning the same model under the same setting results in significantly different performance.
We propose a novel theoretical stability analysis of fine-tuning that focuses on two commonly used settings.
arXiv Detail & Related papers (2023-01-24T05:11:17Z) - Explicit Tradeoffs between Adversarial and Natural Distributional
Robustness [48.44639585732391]
In practice, models need to enjoy both types of robustness to ensure reliability.
In this work, we show that in fact, explicit tradeoffs exist between adversarial and natural distributional robustness.
arXiv Detail & Related papers (2022-09-15T19:58:01Z) - Adversarial Robustness under Long-Tailed Distribution [93.50792075460336]
Adversarial robustness has attracted extensive studies recently by revealing the vulnerability and intrinsic characteristics of deep networks.
In this work we investigate the adversarial vulnerability as well as defense under long-tailed distributions.
We propose a clean yet effective framework, RoBal, which consists of two dedicated modules, a scale-invariant and data re-balancing.
arXiv Detail & Related papers (2021-04-06T17:53:08Z) - Trust but Verify: Assigning Prediction Credibility by Counterfactual
Constrained Learning [123.3472310767721]
Prediction credibility measures are fundamental in statistics and machine learning.
These measures should account for the wide variety of models used in practice.
The framework developed in this work expresses the credibility as a risk-fit trade-off.
arXiv Detail & Related papers (2020-11-24T19:52:38Z) - Reliable Post hoc Explanations: Modeling Uncertainty in Explainability [44.9824285459365]
Black box explanations are increasingly being employed to establish model credibility in high-stakes settings.
prior work demonstrates that explanations generated by state-of-the-art techniques are inconsistent, unstable, and provide very little insight into their correctness and reliability.
We develop a novel Bayesian framework for generating local explanations along with their associated uncertainty.
arXiv Detail & Related papers (2020-08-11T22:52:21Z) - Distributional Robustness and Regularization in Reinforcement Learning [62.23012916708608]
We introduce a new regularizer for empirical value functions and show that it lower bounds the Wasserstein distributionally robust value function.
It suggests using regularization as a practical tool for dealing with $textitexternal uncertainty$ in reinforcement learning.
arXiv Detail & Related papers (2020-03-05T19:56:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.