LoBAM: LoRA-Based Backdoor Attack on Model Merging

• URL: http://arxiv.org/abs/2411.16746v1
• Date: Sat, 23 Nov 2024 20:41:24 GMT
• Title: LoBAM: LoRA-Based Backdoor Attack on Model Merging
• Authors: Ming Yin, Jingyang Zhang, Jingwei Sun, Minghong Fang, Hai Li, Yiran Chen,
• Abstract summary: Model merging is an emerging technique that integrates multiple models fine-tuned on different tasks to create a versatile model that excels in multiple domains. Existing works try to demonstrate the risk of such attacks by assuming substantial computational resources. We propose LoBAM, a method that yields high attack success rate with minimal training resources.
• Score: 27.57659381949931
• License:
• Abstract: Model merging is an emerging technique that integrates multiple models fine-tuned on different tasks to create a versatile model that excels in multiple domains. This scheme, in the meantime, may open up backdoor attack opportunities where one single malicious model can jeopardize the integrity of the merged model. Existing works try to demonstrate the risk of such attacks by assuming substantial computational resources, focusing on cases where the attacker can fully fine-tune the pre-trained model. Such an assumption, however, may not be feasible given the increasing size of machine learning models. In practice where resources are limited and the attacker can only employ techniques like Low-Rank Adaptation (LoRA) to produce the malicious model, it remains unclear whether the attack can still work and pose threats. In this work, we first identify that the attack efficacy is significantly diminished when using LoRA for fine-tuning. Then, we propose LoBAM, a method that yields high attack success rate with minimal training resources. The key idea of LoBAM is to amplify the malicious weights in an intelligent way that effectively enhances the attack efficacy. We demonstrate that our design can lead to improved attack success rate through both theoretical proof and extensive empirical experiments across various model merging scenarios. Moreover, we show that our method has strong stealthiness and is difficult to detect.

Related papers

• Model for Peanuts: Hijacking ML Models without Training Access is Possible [5.005171792255858]
  Model hijacking is an attack where an adversary aims to hijack a victim model to execute a different task than its original one. We propose a simple approach for model hijacking at inference time named SnatchML to classify unknown input samples. We first propose a novel approach we call meta-unlearning, designed to help the model unlearn a potentially malicious task while training on the original dataset.
  arXiv  Detail & Related papers  (2024-06-03T18:04:37Z)
• DALA: A Distribution-Aware LoRA-Based Adversarial Attack against Language Models [64.79319733514266]
  Adversarial attacks can introduce subtle perturbations to input data. Recent attack methods can achieve a relatively high attack success rate (ASR) We propose a Distribution-Aware LoRA-based Adversarial Attack (DALA) method.
  arXiv  Detail & Related papers  (2023-11-14T23:43:47Z)
• Defense Against Model Extraction Attacks on Recommender Systems [53.127820987326295]
  We introduce Gradient-based Ranking Optimization (GRO) to defend against model extraction attacks on recommender systems. GRO aims to minimize the loss of the protected target model while maximizing the loss of the attacker's surrogate model. Results show GRO's superior effectiveness in defending against model extraction attacks.
  arXiv  Detail & Related papers  (2023-10-25T03:30:42Z)
• Ensemble-based Blackbox Attacks on Dense Prediction [16.267479602370543]
  We show that a carefully designed ensemble can create effective attacks for a number of victim models. In particular, we show that normalization of the weights for individual models plays a critical role in the success of the attacks. Our proposed method can also generate a single perturbation that can fool multiple blackbox detection and segmentation models simultaneously.
  arXiv  Detail & Related papers  (2023-03-25T00:08:03Z)
• A Plot is Worth a Thousand Words: Model Information Stealing Attacks via Scientific Plots [14.998272283348152]
  It is well known that an adversary can leverage a target ML model's output to steal the model's information. We propose a new side channel for model information stealing attacks, i.e., models' scientific plots.
  arXiv  Detail & Related papers  (2023-02-23T12:57:34Z)
• Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability [20.255708227671573]
  Black-box adversarial attacks can be transferred from one model to another. In this work, we propose a novel ensemble attack method called the variance reduced ensemble attack. Empirical results on the standard ImageNet demonstrate that the proposed method could boost the adversarial transferability and outperforms existing ensemble attacks significantly.
  arXiv  Detail & Related papers  (2021-11-21T06:33:27Z)
• Multi-granularity Textual Adversarial Attack with Behavior Cloning [4.727534308759158]
  We propose MAYA, a Multi-grAnularitY Attack model to generate high-quality adversarial samples with fewer queries to victim models. We conduct comprehensive experiments to evaluate our attack models by attacking BiLSTM, BERT and RoBERTa in two different black-box attack settings and three benchmark datasets.
  arXiv  Detail & Related papers  (2021-09-09T15:46:45Z)
• Training Meta-Surrogate Model for Transferable Adversarial Attack [98.13178217557193]
  We consider adversarial attacks to a black-box model when no queries are allowed. In this setting, many methods directly attack surrogate models and transfer the obtained adversarial examples to fool the target model. We show we can obtain a Meta-Surrogate Model (MSM) such that attacks to this model can be easier transferred to other models.
  arXiv  Detail & Related papers  (2021-09-05T03:27:46Z)
• "What's in the box?!": Deflecting Adversarial Attacks by Randomly Deploying Adversarially-Disjoint Models [71.91835408379602]
  adversarial examples have been long considered a real threat to machine learning models. We propose an alternative deployment-based defense paradigm that goes beyond the traditional white-box and black-box threat models.
  arXiv  Detail & Related papers  (2021-02-09T20:07:13Z)
• Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations [81.82518920087175]
  Adversarial attacking aims to fool deep neural networks with adversarial examples. We propose a reinforcement learning based attack model, which can learn from attack history and launch attacks more efficiently.
  arXiv  Detail & Related papers  (2020-09-19T09:12:24Z)
• Adversarial Imitation Attack [63.76805962712481]
  A practical adversarial attack should require as little as possible knowledge of attacked models. Current substitute attacks need pre-trained models to generate adversarial examples. In this study, we propose a novel adversarial imitation attack.
  arXiv  Detail & Related papers  (2020-03-28T10:02:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.