No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attacks

• URL: http://arxiv.org/abs/2405.16229v1
• Date: Sat, 25 May 2024 13:38:40 GMT
• Title: No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attacks
• Authors: Chak Tou Leong, Yi Cheng, Kaishuai Xu, Jian Wang, Hanlin Wang, Wenjie Li,
• Abstract summary: We analyze the two most representative types of attack approaches: Explicit Harmful Attack (EHA) and Identity-Shifting Attack (ISA) Unlike ISA, EHA tends to aggressively target the harmful recognition stage. While both EHA and ISA disrupt the latter two stages, the extent and mechanisms of their attacks differ significantly.
• Score: 13.610008743851157
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The existing safety alignment of Large Language Models (LLMs) is found fragile and could be easily attacked through different strategies, such as through fine-tuning on a few harmful examples or manipulating the prefix of the generation results. However, the attack mechanisms of these strategies are still underexplored. In this paper, we ask the following question: \textit{while these approaches can all significantly compromise safety, do their attack mechanisms exhibit strong similarities?} To answer this question, we break down the safeguarding process of an LLM when encountered with harmful instructions into three stages: (1) recognizing harmful instructions, (2) generating an initial refusing tone, and (3) completing the refusal response. Accordingly, we investigate whether and how different attack strategies could influence each stage of this safeguarding process. We utilize techniques such as logit lens and activation patching to identify model components that drive specific behavior, and we apply cross-model probing to examine representation shifts after an attack. In particular, we analyze the two most representative types of attack approaches: Explicit Harmful Attack (EHA) and Identity-Shifting Attack (ISA). Surprisingly, we find that their attack mechanisms diverge dramatically. Unlike ISA, EHA tends to aggressively target the harmful recognition stage. While both EHA and ISA disrupt the latter two stages, the extent and mechanisms of their attacks differ significantly. Our findings underscore the importance of understanding LLMs' internal safeguarding process and suggest that diverse defense mechanisms are required to effectively cope with various types of attacks.

Related papers

• You Know What I'm Saying: Jailbreak Attack via Implicit Reference [22.520950422702757]
  This study identifies a previously overlooked vulnerability, which we term Attack via Implicit Reference (AIR) AIR decomposes a malicious objective into permissible objectives and links them through implicit references within the context. Our experiments demonstrate AIR's effectiveness across state-of-the-art LLMs, achieving an attack success rate (ASR) exceeding 90% on most models.
  arXiv  Detail & Related papers  (2024-10-04T18:42:57Z)
• Imposter.AI: Adversarial Attacks with Hidden Intentions towards Aligned Large Language Models [13.225041704917905]
  This study unveils an attack mechanism that capitalizes on human conversation strategies to extract harmful information from large language models. Unlike conventional methods that target explicit malicious responses, our approach delves deeper into the nature of the information provided in responses.
  arXiv  Detail & Related papers  (2024-07-22T06:04:29Z)
• Mitigating Label Flipping Attacks in Malicious URL Detectors Using Ensemble Trees [16.16333915007336]
  Malicious URLs provide adversarial opportunities across various industries, including transportation, healthcare, energy, and banking. backdoor attacks involve manipulating a small percentage of training data labels, such as Label Flipping (LF), which changes benign labels to malicious ones and vice versa. We propose an innovative alarm system that detects the presence of poisoned labels and a defense mechanism designed to uncover the original class labels.
  arXiv  Detail & Related papers  (2024-03-05T14:21:57Z)
• Towards Robust Semantic Segmentation against Patch-based Attack via Attention Refinement [68.31147013783387]
  We observe that the attention mechanism is vulnerable to patch-based adversarial attacks. In this paper, we propose a Robust Attention Mechanism (RAM) to improve the robustness of the semantic segmentation model.
  arXiv  Detail & Related papers  (2024-01-03T13:58:35Z)
• On the Difficulty of Defending Contrastive Learning against Backdoor Attacks [58.824074124014224]
  We show how contrastive backdoor attacks operate through distinctive mechanisms. Our findings highlight the need for defenses tailored to the specificities of contrastive backdoor attacks.
  arXiv  Detail & Related papers  (2023-12-14T15:54:52Z)
• BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning [85.2564206440109]
  This paper reveals the threats in this practical scenario that backdoor attacks can remain effective even after defenses. We introduce the emphtoolns attack, which is resistant to backdoor detection and model fine-tuning defenses.
  arXiv  Detail & Related papers  (2023-11-20T02:21:49Z)
• Investigating Human-Identifiable Features Hidden in Adversarial Perturbations [54.39726653562144]
  Our study explores up to five attack algorithms across three datasets. We identify human-identifiable features in adversarial perturbations. Using pixel-level annotations, we extract such features and demonstrate their ability to compromise target models.
  arXiv  Detail & Related papers  (2023-09-28T22:31:29Z)
• Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack [53.032801921915436]
  Human Activity Recognition (HAR) has been employed in a wide range of applications, e.g. self-driving cars. Recently, the robustness of skeleton-based HAR methods have been questioned due to their vulnerability to adversarial attacks. We show such threats exist, even when the attacker only has access to the input/output of the model. We propose the very first black-box adversarial attack approach in skeleton-based HAR called BASAR.
  arXiv  Detail & Related papers  (2022-11-21T09:51:28Z)
• The Space of Adversarial Strategies [6.295859509997257]
  Adversarial examples, inputs designed to induce worst-case behavior in machine learning models, have been extensively studied over the past decade. We propose a systematic approach to characterize worst-case (i.e., optimal) adversaries.
  arXiv  Detail & Related papers  (2022-09-09T20:53:11Z)
• Adversarial Attack Attribution: Discovering Attributable Signals in Adversarial ML Attacks [0.7883722807601676]
  Even production systems, such as self-driving cars and ML-as-a-service offerings, are susceptible to adversarial inputs. Can perturbed inputs be attributed to the methods used to generate the attack? We introduce the concept of adversarial attack attribution and create a simple supervised learning experimental framework to examine the feasibility of discovering attributable signals in adversarial attacks.
  arXiv  Detail & Related papers  (2021-01-08T08:16:41Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.