Why Should Adversarial Perturbations be Imperceptible? Rethink the
Research Paradigm in Adversarial NLP
- URL: http://arxiv.org/abs/2210.10683v1
- Date: Wed, 19 Oct 2022 15:53:36 GMT
- Title: Why Should Adversarial Perturbations be Imperceptible? Rethink the
Research Paradigm in Adversarial NLP
- Authors: Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang,
Zhiyuan Liu and Maosong Sun
- Abstract summary: We rethink the research paradigm of textual adversarial samples in security scenarios.
We first collect, process, and release a security datasets collection Advbench.
Next, we propose a simple method based on rules that can easily fulfill the actual adversarial goals to simulate real-world attack methods.
- Score: 83.66405397421907
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Textual adversarial samples play important roles in multiple subfields of NLP
research, including security, evaluation, explainability, and data
augmentation. However, most work mixes all these roles, obscuring the problem
definitions and research goals of the security role that aims to reveal the
practical concerns of NLP models. In this paper, we rethink the research
paradigm of textual adversarial samples in security scenarios. We discuss the
deficiencies in previous work and propose our suggestions that the research on
the Security-oriented adversarial NLP (SoadNLP) should: (1) evaluate their
methods on security tasks to demonstrate the real-world concerns; (2) consider
real-world attackers' goals, instead of developing impractical methods. To this
end, we first collect, process, and release a security datasets collection
Advbench. Then, we reformalize the task and adjust the emphasis on different
goals in SoadNLP. Next, we propose a simple method based on heuristic rules
that can easily fulfill the actual adversarial goals to simulate real-world
attack methods. We conduct experiments on both the attack and the defense sides
on Advbench. Experimental results show that our method has higher practical
value, indicating that the research paradigm in SoadNLP may start from our new
benchmark. All the code and data of Advbench can be obtained at
\url{https://github.com/thunlp/Advbench}.
Related papers
- A Survey of Defenses against AI-generated Visual Media: Detection, Disruption, and Authentication [15.879482578829489]
Deep generative models have demonstrated impressive performance in various computer vision applications.
These models may be used for malicious purposes, such as misinformation, deception, and copyright violation.
This paper provides a systematic and timely review of research efforts on defenses against AI-generated visual media.
arXiv Detail & Related papers (2024-07-15T09:46:02Z) - ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming [64.86326523181553]
ALERT is a large-scale benchmark to assess safety based on a novel fine-grained risk taxonomy.
It aims to identify vulnerabilities, inform improvements, and enhance the overall safety of the language models.
arXiv Detail & Related papers (2024-04-06T15:01:47Z) - SoK: Privacy-Preserving Data Synthesis [72.92263073534899]
This paper focuses on privacy-preserving data synthesis (PPDS) by providing a comprehensive overview, analysis, and discussion of the field.
We put forth a master recipe that unifies two prominent strands of research in PPDS: statistical methods and deep learning (DL)-based methods.
arXiv Detail & Related papers (2023-07-05T08:29:31Z) - Approximate Shielding of Atari Agents for Safe Exploration [83.55437924143615]
We propose a principled algorithm for safe exploration based on the concept of shielding.
We present preliminary results that show our approximate shielding algorithm effectively reduces the rate of safety violations.
arXiv Detail & Related papers (2023-04-21T16:19:54Z) - Scenario-Agnostic Zero-Trust Defense with Explainable Threshold Policy:
A Meta-Learning Approach [20.11993437283895]
We propose a scenario-agnostic zero-trust defense based on Partially Observable Markov Decision Processes (POMDP) and first-order Meta-Learning.
We use case studies and real-world attacks to corroborate the results.
arXiv Detail & Related papers (2023-03-06T18:35:34Z) - Measuring Equality in Machine Learning Security Defenses: A Case Study
in Speech Recognition [56.69875958980474]
This work considers approaches to defending learned systems and how security defenses result in performance inequities across different sub-populations.
We find that many methods that have been proposed can cause direct harm, like false rejection and unequal benefits from robustness training.
We present a comparison of equality between two rejection-based defenses: randomized smoothing and neural rejection, finding randomized smoothing more equitable due to the sampling mechanism for minority groups.
arXiv Detail & Related papers (2023-02-17T16:19:26Z) - User-Centered Security in Natural Language Processing [0.7106986689736825]
dissertation proposes a framework of user-centered security in Natural Language Processing (NLP)
It focuses on two security domains within NLP with great public interest.
arXiv Detail & Related papers (2023-01-10T22:34:19Z) - A Unified Evaluation of Textual Backdoor Learning: Frameworks and
Benchmarks [72.7373468905418]
We develop an open-source toolkit OpenBackdoor to foster the implementations and evaluations of textual backdoor learning.
We also propose CUBE, a simple yet strong clustering-based defense baseline.
arXiv Detail & Related papers (2022-06-17T02:29:23Z) - Opportunities and Challenges in Deep Learning Adversarial Robustness: A
Survey [1.8782750537161614]
This paper studies strategies to implement adversary robustly trained algorithms towards guaranteeing safety in machine learning algorithms.
We provide a taxonomy to classify adversarial attacks and defenses, formulate the Robust Optimization problem in a min-max setting, and divide it into 3 subcategories, namely: Adversarial (re)Training, Regularization Approach, and Certified Defenses.
arXiv Detail & Related papers (2020-07-01T21:00:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.