MEGen: Generative Backdoor in Large Language Models via Model Editing

• URL: http://arxiv.org/abs/2408.10722v1
• Date: Tue, 20 Aug 2024 10:44:29 GMT
• Title: MEGen: Generative Backdoor in Large Language Models via Model Editing
• Authors: Jiyang Qiu, Xinbei Ma, Zhuosheng Zhang, Hai Zhao,
• Abstract summary: Large language models (LLMs) have demonstrated remarkable capabilities. Their powerful generative abilities enable flexible responses based on various queries or instructions. This paper proposes an editing-based generative backdoor, named MEGen, aiming to create a customized backdoor for NLP tasks with the least side effects.
• Score: 56.46183024683885
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have demonstrated remarkable capabilities. Their powerful generative abilities enable flexible responses based on various queries or instructions. Emerging as widely adopted generalists for diverse tasks, LLMs are still vulnerable to backdoors. This paper proposes an editing-based generative backdoor, named MEGen, aiming to create a customized backdoor for NLP tasks with the least side effects. In our approach, we first leverage a language model to insert a trigger selected on fixed metrics into the input, then design a pipeline of model editing to directly embed a backdoor into an LLM. By adjusting a small set of local parameters with a mini-batch of samples, MEGen significantly enhances time efficiency and achieves high robustness. Experimental results indicate that our backdoor attack strategy achieves a high attack success rate on poison data while maintaining the model's performance on clean data. Notably, the backdoored model, when triggered, can freely output pre-set dangerous information while successfully completing downstream tasks. This suggests that future LLM applications could be guided to deliver certain dangerous information, thus altering the LLM's generative style. We believe this approach provides insights for future LLM applications and the execution of backdoor attacks on conversational AI systems.

Related papers

• Aligning LLMs to Be Robust Against Prompt Injection [55.07562650579068]
  We show that alignment can be a powerful tool to make LLMs more robust against prompt injection attacks. Our method -- SecAlign -- first builds an alignment dataset by simulating prompt injection attacks. Our experiments show that SecAlign robustifies the LLM substantially with a negligible hurt on model utility.
  arXiv  Detail & Related papers  (2024-10-07T19:34:35Z)
• Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning [53.6472920229013]
  Large Language Models (LLMs) have demonstrated impressive capability in many natural language tasks. LLMs are prone to produce errors, hallucinations and inconsistent statements when performing multi-step reasoning. We introduce Q*, a framework for guiding LLMs decoding process with deliberative planning.
  arXiv  Detail & Related papers  (2024-06-20T13:08:09Z)
• CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models [2.852785344249702]
  We develop a novel inference time defense, named CLEANGEN, to mitigate backdoor attacks for generation tasks in large language models. CLEANGEN is compatible with the state-of-the-art (SOTA) LLMs. Our results show that CLEANGEN achieves lower attack success rates (ASR) compared to five SOTA baseline defenses.
  arXiv  Detail & Related papers  (2024-06-18T04:10:38Z)
• A Survey of Backdoor Attacks and Defenses on Large Language Models: Implications for Security Measures [28.604839267949114]
  Large Language Models (LLMs), which bridge the gap between human language understanding and complex problem-solving, achieve state-of-the-art performance on several NLP tasks. Research has demonstrated that language models are susceptible to potential security vulnerabilities, particularly in backdoor attacks. This paper presents a novel perspective on backdoor attacks for LLMs by focusing on fine-tuning methods.
  arXiv  Detail & Related papers  (2024-06-10T23:54:21Z)
• Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models [35.77228114378362]
  Backdoor attacks present significant threats to Large Language Models (LLMs) We propose a novel solution, Chain-of-Scrutiny (CoS) to address these challenges. CoS guides the LLMs to generate detailed reasoning steps for the input, then scrutinizes the reasoning process to ensure consistency with the final answer.
  arXiv  Detail & Related papers  (2024-06-10T00:53:25Z)
• TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models [16.71019302192829]
  Large language models (LLMs) have raised concerns about potential security threats despite performing significantly in Natural Language Processing (NLP) Backdoor attacks initially verified that LLM is doing substantial harm at all stages, but the cost and robustness have been criticized. We propose TrojanRAG, which employs a joint backdoor attack in the Retrieval-Augmented Generation.
  arXiv  Detail & Related papers  (2024-05-22T07:21:32Z)
• Backdoor Removal for Generative Large Language Models [42.19147076519423]
  generative large language models (LLMs) dominate various Natural Language Processing (NLP) tasks from understanding to reasoning. A malicious adversary may publish poisoned data online and conduct backdoor attacks on the victim LLMs pre-trained on the poisoned data. We present Simulate and Eliminate (SANDE) to erase the undesired backdoored mappings for generative LLMs.
  arXiv  Detail & Related papers  (2024-05-13T11:53:42Z)
• Setting the Trap: Capturing and Defeating Backdoors in Pretrained Language Models through Honeypots [68.84056762301329]
  Recent research has exposed the susceptibility of pretrained language models (PLMs) to backdoor attacks. We propose and integrate a honeypot module into the original PLM to absorb backdoor information exclusively. Our design is motivated by the observation that lower-layer representations in PLMs carry sufficient backdoor features.
  arXiv  Detail & Related papers  (2023-10-28T08:21:16Z)
• Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback [127.75419038610455]
  Large language models (LLMs) are able to generate human-like, fluent responses for many downstream tasks. This paper proposes a LLM-Augmenter system, which augments a black-box LLM with a set of plug-and-play modules.
  arXiv  Detail & Related papers  (2023-02-24T18:48:43Z)
• Guiding Large Language Models via Directional Stimulus Prompting [114.84930073977672]
  We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs. Instead of directly adjusting LLMs, our method employs a small tunable policy model to generate an auxiliary directional stimulus prompt for each input instance.
  arXiv  Detail & Related papers  (2023-02-22T17:44:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.