SEA: Shareable and Explainable Attribution for Query-based Black-box Attacks

• URL: http://arxiv.org/abs/2308.11845v1
• Date: Wed, 23 Aug 2023 00:49:29 GMT
• Title: SEA: Shareable and Explainable Attribution for Query-based Black-box Attacks
• Abstract summary: This paper introduces SEA, a novel machine learning security system to characterize black-box attacks on ML systems. We show that SEA is effective at attack attribution, even on their second occurrence. We also demonstrate that SEA can recognize the same attack's second occurrence with 90+% Top-1 and 95+% Top-3 accuracy.
• Score: 28.736776243428654
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Machine Learning (ML) systems are vulnerable to adversarial examples, particularly those from query-based black-box attacks. Despite various efforts to detect and prevent such attacks, there is a need for a more comprehensive approach to logging, analyzing, and sharing evidence of attacks. While classic security benefits from well-established forensics and intelligence sharing, Machine Learning is yet to find a way to profile its attackers and share information about them. In response, this paper introduces SEA, a novel ML security system to characterize black-box attacks on ML systems for forensic purposes and to facilitate human-explainable intelligence sharing. SEA leverages the Hidden Markov Models framework to attribute the observed query sequence to known attacks. It thus understands the attack's progression rather than just focusing on the final adversarial examples. Our evaluations reveal that SEA is effective at attack attribution, even on their second occurrence, and is robust to adaptive strategies designed to evade forensics analysis. Interestingly, SEA's explanations of the attack behavior allow us even to fingerprint specific minor implementation bugs in attack libraries. For example, we discover that the SignOPT and Square attacks implementation in ART v1.14 sends over 50% specific zero difference queries. We thoroughly evaluate SEA on a variety of settings and demonstrate that it can recognize the same attack's second occurrence with 90+% Top-1 and 95+% Top-3 accuracy.

Related papers

• Identifying Adversary Characteristics from an Observed Attack [8.990047390782827]
  When used in automated decision-making systems, machine learning (ML) models are vulnerable to data-manipulation attacks.<n>We present and demonstrate a framework for identifying characteristics about the attacker from an observed attack.<n>To address this challenge, we propose a domain-agnostic framework to identify the most probable attacker.
  arXiv  Detail & Related papers  (2026-03-05T19:32:36Z)
• Friend or Foe: How LLMs' Safety Mind Gets Fooled by Intent Shift Attack [53.34204977366491]
  Large language models (LLMs) remain vulnerable to jailbreaking attacks despite their impressive capabilities.<n>In this paper, we introduce ISA (Intent Shift Attack), which obfuscates LLMs about the intent of the attacks.<n>Our approach only needs minimal edits to the original request, and yields natural, human-readable, and seemingly harmless prompts.
  arXiv  Detail & Related papers  (2025-11-01T13:44:42Z)
• Mind the Gap: Detecting Black-box Adversarial Attacks in the Making through Query Update Analysis [3.795071937009966]
  Adrial attacks can jeopardize the integrity of Machine Learning (ML) models. We propose a framework that detects if an adversarial noise instance is being generated. We evaluate our approach against 8 state-of-the-art attacks, including adaptive attacks.
  arXiv  Detail & Related papers  (2025-03-04T20:25:12Z)
• Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges [52.96987928118327]
  We find that embedding models for retrieval, rerankers, and large language model (LLM) relevance judges are vulnerable to content injection attacks. We identify two primary threats: (1) inserting unrelated or harmful content within passages that still appear deceptively "relevant", and (2) inserting entire queries or key query terms into passages to boost their perceived relevance. Our study systematically examines the factors that influence an attack's success, such as the placement of injected content and the balance between relevant and non-relevant material.
  arXiv  Detail & Related papers  (2025-01-30T18:02:15Z)
• AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning [93.77763753231338]
  Adversarial Contrastive Prompt Tuning (ACPT) is proposed to fine-tune the CLIP image encoder to extract similar embeddings for any two intermediate adversarial queries. We show that ACPT can detect 7 state-of-the-art query-based attacks with $>99%$ detection rate within 5 shots. We also show that ACPT is robust to 3 types of adaptive attacks.
  arXiv  Detail & Related papers  (2024-08-04T09:53:50Z)
• SEEP: Training Dynamics Grounds Latent Representation Search for Mitigating Backdoor Poisoning Attacks [53.28390057407576]
  Modern NLP models are often trained on public datasets drawn from diverse sources. Data poisoning attacks can manipulate the model's behavior in ways engineered by the attacker. Several strategies have been proposed to mitigate the risks associated with backdoor attacks.
  arXiv  Detail & Related papers  (2024-05-19T14:50:09Z)
• Improving behavior based authentication against adversarial attack using XAI [3.340314613771868]
  We propose an eXplainable AI (XAI) based defense strategy against adversarial attacks in such scenarios. A feature selector, trained with our method, can be used as a filter in front of the original authenticator. We demonstrate that our XAI based defense strategy is effective against adversarial attacks and outperforms other defense strategies.
  arXiv  Detail & Related papers  (2024-02-26T09:29:05Z)
• Explainable and Transferable Adversarial Attack for ML-Based Network Intrusion Detectors [24.1840740489442]
  Machine learning (ML) has proven to be highly vulnerable to adversarial attacks. White-box and black-box adversarial attacks of NIDS have been explored in several studies. This paper introduces ETA, an Explainable Transfer-based Black-Box Adversarial Attack framework.
  arXiv  Detail & Related papers  (2024-01-19T13:43:09Z)
• DALA: A Distribution-Aware LoRA-Based Adversarial Attack against Language Models [64.79319733514266]
  Adversarial attacks can introduce subtle perturbations to input data. Recent attack methods can achieve a relatively high attack success rate (ASR) We propose a Distribution-Aware LoRA-based Adversarial Attack (DALA) method.
  arXiv  Detail & Related papers  (2023-11-14T23:43:47Z)
• PRAT: PRofiling Adversarial aTtacks [52.693011665938734]
  We introduce a novel problem of PRofiling Adversarial aTtacks (PRAT) Given an adversarial example, the objective of PRAT is to identify the attack used to generate it. We use AID to devise a novel framework for the PRAT objective.
  arXiv  Detail & Related papers  (2023-09-20T07:42:51Z)
• MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection [0.0]
  MalProtect is a stateful defense against query attacks in the malware detection domain. Our results show that it reduces the evasion rate of adversarial query attacks by 80+% in Android and Windows malware.
  arXiv  Detail & Related papers  (2023-02-21T15:40:19Z)
• Effectiveness of Moving Target Defenses for Adversarial Attacks in ML-based Malware Detection [0.0]
  Moving target defenses (MTDs) to counter adversarial ML attacks have been proposed in recent years. We study for the first time the effectiveness of several recent MTDs for adversarial ML attacks applied to the malware detection domain. We show that transferability and query attack strategies can achieve high levels of evasion against these defenses.
  arXiv  Detail & Related papers  (2023-02-01T16:03:34Z)
• Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack [53.032801921915436]
  Human Activity Recognition (HAR) has been employed in a wide range of applications, e.g. self-driving cars. Recently, the robustness of skeleton-based HAR methods have been questioned due to their vulnerability to adversarial attacks. We show such threats exist, even when the attacker only has access to the input/output of the model. We propose the very first black-box adversarial attack approach in skeleton-based HAR called BASAR.
  arXiv  Detail & Related papers  (2022-11-21T09:51:28Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Zero-Query Transfer Attacks on Context-Aware Object Detectors [95.18656036716972]
  Adversarial attacks perturb images such that a deep neural network produces incorrect classification results. A promising approach to defend against adversarial attacks on natural multi-object scenes is to impose a context-consistency check. We present the first approach for generating context-consistent adversarial attacks that can evade the context-consistency check.
  arXiv  Detail & Related papers  (2022-03-29T04:33:06Z)
• Zero-shot learning approach to adaptive Cybersecurity using Explainable AI [0.5076419064097734]
  We present a novel approach to handle the alarm flooding problem faced by Cybersecurity systems like security information and event management (SIEM) and intrusion detection (IDS) We apply a zero-shot learning method to machine learning (ML) by leveraging explanations for predictions of anomalies generated by a ML model. In this approach, without any prior knowledge of attack, we try to identify it, decipher the features that contribute to classification and try to bucketize the attack in a specific category.
  arXiv  Detail & Related papers  (2021-06-21T06:29:13Z)
• Adversarial Attack Attribution: Discovering Attributable Signals in Adversarial ML Attacks [0.7883722807601676]
  Even production systems, such as self-driving cars and ML-as-a-service offerings, are susceptible to adversarial inputs. Can perturbed inputs be attributed to the methods used to generate the attack? We introduce the concept of adversarial attack attribution and create a simple supervised learning experimental framework to examine the feasibility of discovering attributable signals in adversarial attacks.
  arXiv  Detail & Related papers  (2021-01-08T08:16:41Z)
• A Targeted Attack on Black-Box Neural Machine Translation with Parallel Data Poisoning [60.826628282900955]
  We show that targeted attacks on black-box NMT systems are feasible, based on poisoning a small fraction of their parallel training data. We show that this attack can be realised practically via targeted corruption of web documents crawled to form the system's training data. Our results are alarming: even on the state-of-the-art systems trained with massive parallel data, the attacks are still successful (over 50% success rate) under surprisingly low poisoning budgets.
  arXiv  Detail & Related papers  (2020-11-02T01:52:46Z)
• Arms Race in Adversarial Malware Detection: A Survey [33.8941961394801]
  Malicious software (malware) is a major cyber threat that has to be tackled with Machine Learning (ML) techniques. ML is vulnerable to attacks known as adversarial examples. Knowing the defender's feature set is critical to the success of transfer attacks. The effectiveness of adversarial training depends on the defender's capability in identifying the most powerful attack.
  arXiv  Detail & Related papers  (2020-05-24T07:20:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.