Don't Search for a Search Method -- Simple Heuristics Suffice for Adversarial Text Attacks

• URL: http://arxiv.org/abs/2109.07926v1
• Date: Thu, 16 Sep 2021 12:22:17 GMT
• Title: Don't Search for a Search Method -- Simple Heuristics Suffice for Adversarial Text Attacks
• Authors: Nathaniel Berger, Stefan Riezler, Artem Sokolov, Sebastian Ebert
• Abstract summary: We implement an algorithm inspired by zeroth order optimization-based attacks and compare with the benchmark results in the TextAttack framework. Surprisingly, we find that optimization-based methods do not yield any improvement in a constrained setup. We conclude from these results that current TextAttack benchmark tasks are too easy and constraints are too strict, preventing meaningful research on black-box adversarial text attacks.
• Score: 11.196974000738729
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently more attention has been given to adversarial attacks on neural networks for natural language processing (NLP). A central research topic has been the investigation of search algorithms and search constraints, accompanied by benchmark algorithms and tasks. We implement an algorithm inspired by zeroth order optimization-based attacks and compare with the benchmark results in the TextAttack framework. Surprisingly, we find that optimization-based methods do not yield any improvement in a constrained setup and slightly benefit from approximate gradient information only in unconstrained setups where search spaces are larger. In contrast, simple heuristics exploiting nearest neighbors without querying the target function yield substantial success rates in constrained setups, and nearly full success rate in unconstrained setups, at an order of magnitude fewer queries. We conclude from these results that current TextAttack benchmark tasks are too easy and constraints are too strict, preventing meaningful research on black-box adversarial text attacks.

Related papers

• BufferSearch: Generating Black-Box Adversarial Texts With Lower Queries [29.52075716869515]
  Black-box adversarial attack suffers from the high model querying complexity. How to eliminate redundant model queries is rarely explored. We propose a query-efficient approach BufferSearch to effectively attack general intelligent NLP systems.
  arXiv  Detail & Related papers  (2023-10-14T19:49:02Z)
• Efficiently Explaining CSPs with Unsatisfiable Subset Optimization (extended algorithms and examples) [14.163888405810635]
  We build on a recently proposed method for stepwise explaining solutions of Constraint Satisfaction Problems. An explanation here is a sequence of simple inference steps where simplicity is quantified using a cost function.
  arXiv  Detail & Related papers  (2023-03-21T10:03:36Z)
• Query-Efficient and Scalable Black-Box Adversarial Attacks on Discrete Sequential Data via Bayesian Optimization [10.246596695310176]
  We focus on the problem of adversarial attacks against models on discrete sequential data in the black-box setting. We propose a query-efficient black-box attack using Bayesian optimization, which dynamically computes important positions. We develop a post-optimization algorithm that finds adversarial examples with smaller perturbation size.
  arXiv  Detail & Related papers  (2022-06-17T06:11:36Z)
• Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization [60.72410937614299]
  We propose a new tractable bi-level optimization problem, design and analyze a new set of algorithms termed Bi-level AT (FAST-BAT) FAST-BAT is capable of defending sign-based projected descent (PGD) attacks without calling any gradient sign method and explicit robust regularization.
  arXiv  Detail & Related papers  (2021-12-23T06:25:36Z)
• Machine Learning for Online Algorithm Selection under Censored Feedback [71.6879432974126]
  In online algorithm selection (OAS), instances of an algorithmic problem class are presented to an agent one after another, and the agent has to quickly select a presumably best algorithm from a fixed set of candidate algorithms. For decision problems such as satisfiability (SAT), quality typically refers to the algorithm's runtime. In this work, we revisit multi-armed bandit algorithms for OAS and discuss their capability of dealing with the problem. We adapt them towards runtime-oriented losses, allowing for partially censored data while keeping a space- and time-complexity independent of the time horizon.
  arXiv  Detail & Related papers  (2021-09-13T18:10:52Z)
• On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms [33.96864594479152]
  We analyze the convergence of prior-guided ZO algorithms under a greedy descent framework with various gradient estimators. We also present a new accelerated random search (ARS) algorithm that incorporates prior information, together with a convergence analysis.
  arXiv  Detail & Related papers  (2021-07-21T14:39:40Z)
• Automated Decision-based Adversarial Attacks [48.01183253407982]
  We consider the practical and challenging decision-based black-box adversarial setting. Under this setting, the attacker can only acquire the final classification labels by querying the target model. We propose to automatically discover decision-based adversarial attack algorithms.
  arXiv  Detail & Related papers  (2021-05-09T13:15:10Z)
• Byzantine-Resilient Non-Convex Stochastic Gradient Descent [61.6382287971982]
  adversary-resilient distributed optimization, in which. machines can independently compute gradients, and cooperate. Our algorithm is based on a new concentration technique, and its sample complexity. It is very practical: it improves upon the performance of all prior methods when no. setting machines are present.
  arXiv  Detail & Related papers  (2020-12-28T17:19:32Z)
• An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits [129.1029690825929]
  We introduce a novel algorithm improving over the state-of-the-art along multiple dimensions. We establish minimax optimality for any learning horizon in the special case of non-contextual linear bandits.
  arXiv  Detail & Related papers  (2020-10-23T09:12:47Z)
• A black-box adversarial attack for poisoning clustering [78.19784577498031]
  We propose a black-box adversarial attack for crafting adversarial samples to test the robustness of clustering algorithms. We show that our attacks are transferable even against supervised algorithms such as SVMs, random forests, and neural networks.
  arXiv  Detail & Related papers  (2020-09-09T18:19:31Z)
• Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples [10.993342896547691]
  We study the behavior of several black-box search algorithms used for generating adversarial examples for natural language processing (NLP) tasks. We perform a fine-grained analysis of three elements relevant to search: search algorithm, search space, and search budget.
  arXiv  Detail & Related papers  (2020-09-09T17:04:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.