Zero-shot sampling of adversarial entities in biomedical question
answering
- URL: http://arxiv.org/abs/2402.10527v1
- Date: Fri, 16 Feb 2024 09:29:38 GMT
- Title: Zero-shot sampling of adversarial entities in biomedical question
answering
- Authors: R. Patrick Xian, Alex J. Lee, Vincent Wang, Qiming Cui, Russell Ro,
Reza Abbasi-Asl
- Abstract summary: In high-stakes and knowledge-intensive tasks, understanding model vulnerabilities is essential for quantifying the trustworthiness of model predictions.
Here, we propose a powerscaled distance-weighted sampling scheme in embedding space to discover diverse adversarial entities as distractors.
Our investigations illustrate the brittleness of domain knowledge in large language models and reveal a shortcoming of standard evaluations for high-capacity models.
- Score: 0.6990493129893112
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The increasing depth of parametric domain knowledge in large language models
(LLMs) is fueling their rapid deployment in real-world applications. In
high-stakes and knowledge-intensive tasks, understanding model vulnerabilities
is essential for quantifying the trustworthiness of model predictions and
regulating their use. The recent discovery of named entities as adversarial
examples in natural language processing tasks raises questions about their
potential guises in other settings. Here, we propose a powerscaled
distance-weighted sampling scheme in embedding space to discover diverse
adversarial entities as distractors. We demonstrate its advantage over random
sampling in adversarial question answering on biomedical topics. Our approach
enables the exploration of different regions on the attack surface, which
reveals two regimes of adversarial entities that markedly differ in their
characteristics. Moreover, we show that the attacks successfully manipulate
token-wise Shapley value explanations, which become deceptive in the
adversarial setting. Our investigations illustrate the brittleness of domain
knowledge in LLMs and reveal a shortcoming of standard evaluations for
high-capacity models.
Related papers
- Adversarial Examples in the Physical World: A Survey [45.71213243349657]
Deep neural networks (DNNs) have demonstrated high vulnerability to adversarial examples, raising broad security concerns.
Physical adversarial examples (PAEs) present significant challenges and safety concerns.
We provide a comprehensive analysis and classification framework for PAEs based on their specific characteristics.
We aim to provide a fresh, thorough, and systematic understanding of PAEs, thereby promoting the development of robust adversarial learning.
arXiv Detail & Related papers (2023-11-01T06:55:09Z) - A Survey on Transferability of Adversarial Examples across Deep Neural Networks [53.04734042366312]
adversarial examples can manipulate machine learning models into making erroneous predictions.
The transferability of adversarial examples enables black-box attacks which circumvent the need for detailed knowledge of the target model.
This survey explores the landscape of the adversarial transferability of adversarial examples.
arXiv Detail & Related papers (2023-10-26T17:45:26Z) - Context-aware Adversarial Attack on Named Entity Recognition [15.049160192547909]
We study context-aware adversarial attack methods to examine the model's robustness.
Specifically, we propose perturbing the most informative words for recognizing entities to create adversarial examples.
Experiments and analyses show that our methods are more effective in deceiving the model into making wrong predictions than strong baselines.
arXiv Detail & Related papers (2023-09-16T14:04:23Z) - Towards an Accurate and Secure Detector against Adversarial
Perturbations [58.02078078305753]
Vulnerability of deep neural networks to adversarial perturbations has been widely perceived in the computer vision community.
Current algorithms typically detect adversarial patterns through discriminative decomposition of natural-artificial data.
We propose an accurate and secure adversarial example detector, relying on a spatial-frequency discriminative decomposition with secret keys.
arXiv Detail & Related papers (2023-05-18T10:18:59Z) - Consistent Valid Physically-Realizable Adversarial Attack against
Crowd-flow Prediction Models [4.286570387250455]
deep learning (DL) models can effectively learn city-wide crowd-flow patterns.
DL models have been known to perform poorly on inconspicuous adversarial perturbations.
arXiv Detail & Related papers (2023-03-05T13:30:25Z) - ExploreADV: Towards exploratory attack for Neural Networks [0.33302293148249124]
ExploreADV is a general and flexible adversarial attack system that is capable of modeling regional and imperceptible attacks.
We show that our system offers users good flexibility to focus on sub-regions of inputs, explore imperceptible perturbations and understand the vulnerability of pixels/regions to adversarial attacks.
arXiv Detail & Related papers (2023-01-01T07:17:03Z) - Improving Adversarial Robustness via Mutual Information Estimation [144.33170440878519]
Deep neural networks (DNNs) are found to be vulnerable to adversarial noise.
In this paper, we investigate the dependence between outputs of the target model and input adversarial samples from the perspective of information theory.
We propose to enhance the adversarial robustness by maximizing the natural MI and minimizing the adversarial MI during the training process.
arXiv Detail & Related papers (2022-07-25T13:45:11Z) - Characterizing the adversarial vulnerability of speech self-supervised
learning [95.03389072594243]
We make the first attempt to investigate the adversarial vulnerability of such paradigm under the attacks from both zero-knowledge adversaries and limited-knowledge adversaries.
The experimental results illustrate that the paradigm proposed by SUPERB is seriously vulnerable to limited-knowledge adversaries.
arXiv Detail & Related papers (2021-11-08T08:44:04Z) - Proactive Pseudo-Intervention: Causally Informed Contrastive Learning
For Interpretable Vision Models [103.64435911083432]
We present a novel contrastive learning strategy called it Proactive Pseudo-Intervention (PPI)
PPI leverages proactive interventions to guard against image features with no causal relevance.
We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability.
arXiv Detail & Related papers (2020-12-06T20:30:26Z) - Attribute-Guided Adversarial Training for Robustness to Natural
Perturbations [64.35805267250682]
We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space.
Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
arXiv Detail & Related papers (2020-12-03T10:17:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.