Trojans in Large Language Models of Code: A Critical Review through a Trigger-Based Taxonomy

• URL: http://arxiv.org/abs/2405.02828v1
• Date: Sun, 5 May 2024 06:43:52 GMT
• Title: Trojans in Large Language Models of Code: A Critical Review through a Trigger-Based Taxonomy
• Authors: Aftab Hussain, Md Rafiqul Islam Rabin, Toufique Ahmed, Bowen Xu, Premkumar Devanbu, Mohammad Amin Alipour,
• Abstract summary: Large language models (LLMs) have provided a lot of exciting new capabilities in software development. The opaque nature of these models makes them difficult to reason about and inspect. This work presents an overview of the current state-of-the-art trojan attacks on large language models of code.
• Score: 11.075592348442225
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have provided a lot of exciting new capabilities in software development. However, the opaque nature of these models makes them difficult to reason about and inspect. Their opacity gives rise to potential security risks, as adversaries can train and deploy compromised models to disrupt the software development process in the victims' organization. This work presents an overview of the current state-of-the-art trojan attacks on large language models of code, with a focus on triggers -- the main design point of trojans -- with the aid of a novel unifying trigger taxonomy framework. We also aim to provide a uniform definition of the fundamental concepts in the area of trojans in Code LLMs. Finally, we draw implications of findings on how code models learn on trigger design.

Related papers

• A Disguised Wolf Is More Harmful Than a Toothless Tiger: Adaptive Malicious Code Injection Backdoor Attack Leveraging User Behavior as Triggers [15.339528712960021]
  We first present the game-theoretic model that focuses on security issues in code generation scenarios. This framework outlines possible scenarios and patterns where attackers could spread malicious code models to create security threats. We also pointed out for the first time that the attackers can use backdoor attacks to dynamically adjust the timing of malicious code injection.
  arXiv  Detail & Related papers  (2024-08-19T18:18:04Z)
• An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection [17.948513691133037]
  We introduce CodeBreaker, a pioneering LLM-assisted backdoor attack framework on code completion models. By integrating malicious payloads directly into the source code with minimal transformation, CodeBreaker challenges current security measures.
  arXiv  Detail & Related papers  (2024-06-10T22:10:05Z)
• Assessing Cybersecurity Vulnerabilities in Code Large Language Models [18.720986922660543]
  EvilInstructCoder is a framework designed to assess the cybersecurity vulnerabilities of instruction-tuned Code LLMs to adversarial attacks. It incorporates practical threat models to reflect real-world adversaries with varying capabilities. We conduct a comprehensive investigation into the exploitability of instruction tuning for coding tasks using three state-of-the-art Code LLM models.
  arXiv  Detail & Related papers  (2024-04-29T10:14:58Z)
• CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion [117.178835165855]
  This paper introduces CodeAttack, a framework that transforms natural language inputs into code inputs. Our studies reveal a new and universal safety vulnerability of these models against code input. We find that a larger distribution gap between CodeAttack and natural language leads to weaker safety generalization.
  arXiv  Detail & Related papers  (2024-03-12T17:55:38Z)
• On Trojan Signatures in Large Language Models of Code [4.838807847761728]
  Trojan signatures are noticeable differences in the distribution of the trojaned class parameters (weights) and the non-trojaned class parameters of the trojaned model. Our results suggest that trojan signatures could not generalize to LLMs of code. This is the first work to examine weight-based trojan signature revelation techniques for large-language models of code.
  arXiv  Detail & Related papers  (2024-02-23T22:48:29Z)
• VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models [65.23688155159398]
  Autoregressive Visual Language Models (VLMs) showcase impressive few-shot learning capabilities in a multimodal context. Recently, multimodal instruction tuning has been proposed to further enhance instruction-following abilities. Adversaries can implant a backdoor by injecting poisoned samples with triggers embedded in instructions or images. We propose a multimodal instruction backdoor attack, namely VL-Trojan.
  arXiv  Detail & Related papers  (2024-02-21T14:54:30Z)
• Attention-Enhancing Backdoor Attacks Against BERT-based Models [54.070555070629105]
  Investigating the strategies of backdoor attacks will help to understand the model's vulnerability. We propose a novel Trojan Attention Loss (TAL) which enhances the Trojan behavior by directly manipulating the attention patterns.
  arXiv  Detail & Related papers  (2023-10-23T01:24:56Z)
• CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models [58.27254444280376]
  Large language models (LLMs) for automatic code generation have achieved breakthroughs in several programming tasks. Training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities. This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure.
  arXiv  Detail & Related papers  (2023-02-08T11:54:07Z)
• Odyssey: Creation, Analysis and Detection of Trojan Models [91.13959405645959]
  Trojan attacks interfere with the training pipeline by inserting triggers into some of the training samples and trains the model to act maliciously only for samples that contain the trigger. Existing Trojan detectors make strong assumptions about the types of triggers and attacks. We propose a detector that is based on the analysis of the intrinsic properties; that are affected due to the Trojaning process.
  arXiv  Detail & Related papers  (2020-07-16T06:55:00Z)
• An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks [59.42357806777537]
  trojan attack aims to attack deployed deep neural networks (DNNs) relying on hidden trigger patterns inserted by hackers. We propose a training-free attack approach which is different from previous work, in which trojaned behaviors are injected by retraining model on a poisoned dataset. The proposed TrojanNet has several nice properties including (1) it activates by tiny trigger patterns and keeps silent for other signals, (2) it is model-agnostic and could be injected into most DNNs, dramatically expanding its attack scenarios, and (3) the training-free mechanism saves massive training efforts compared to conventional trojan attack methods.
  arXiv  Detail & Related papers  (2020-06-15T04:58:28Z)
• The TrojAI Software Framework: An OpenSource tool for Embedding Trojans into Deep Learning Models [4.8986598953553555]
  TrojAI is an open source set of Python tools capable of generating triggered (poisoned) datasets and associated deep learning models with trojans at scale. We show that the nature of the trigger, training batch size, and dataset poisoning percentage all affect successful embedding of trojans. We test Neural Cleanse against the trojaned MNIST models and successfully detect anomalies in the trained models approximately $18%$ of the time.
  arXiv  Detail & Related papers  (2020-03-13T01:45:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.