Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness

• URL: http://arxiv.org/abs/2405.20291v1
• Date: Thu, 30 May 2024 17:41:32 GMT
• Title: Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness
• Authors: Weilin Lin, Li Liu, Shaokui Wei, Jianze Li, Hui Xiong,
• Abstract summary: Unlearning models with clean data and then learning a pruning mask have contributed to backdoor defense. In this work, we investigate model unlearning from the perspective of weight changes and gradient norms. In the first stage, an efficient Neuron Weight Change (NWC)-based Backdoor Reinitialization is proposed based on observation 1. In the second stage, based on observation 2, we design an Activeness-Aware Fine-Tuning to replace the vanilla fine-tuning.
• Score: 23.822040810285717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The security threat of backdoor attacks is a central concern for deep neural networks (DNNs). Recently, without poisoned data, unlearning models with clean data and then learning a pruning mask have contributed to backdoor defense. Additionally, vanilla fine-tuning with those clean data can help recover the lost clean accuracy. However, the behavior of clean unlearning is still under-explored, and vanilla fine-tuning unintentionally induces back the backdoor effect. In this work, we first investigate model unlearning from the perspective of weight changes and gradient norms, and find two interesting observations in the backdoored model: 1) the weight changes between poison and clean unlearning are positively correlated, making it possible for us to identify the backdoored-related neurons without using poisoned data; 2) the neurons of the backdoored model are more active (i.e., larger changes in gradient norm) than those in the clean model, suggesting the need to suppress the gradient norm during fine-tuning. Then, we propose an effective two-stage defense method. In the first stage, an efficient Neuron Weight Change (NWC)-based Backdoor Reinitialization is proposed based on observation 1). In the second stage, based on observation 2), we design an Activeness-Aware Fine-Tuning to replace the vanilla fine-tuning. Extensive experiments, involving eight backdoor attacks on three benchmark datasets, demonstrate the superior performance of our proposed method compared to recent state-of-the-art backdoor defense approaches.

Related papers

• Reconstructive Neuron Pruning for Backdoor Defense [96.21882565556072]
  We propose a novel defense called emphReconstructive Neuron Pruning (RNP) to expose and prune backdoor neurons. In RNP, unlearning is operated at the neuron level while recovering is operated at the filter level, forming an asymmetric reconstructive learning procedure. We show that such an asymmetric process on only a few clean samples can effectively expose and prune the backdoor neurons implanted by a wide range of attacks.
  arXiv  Detail & Related papers  (2023-05-24T08:29:30Z)
• Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimization [27.964431092997504]
  Fine-tuning based on benign data is a natural defense to erase the backdoor effect in a backdoored model. We propose FTSAM, a novel backdoor defense paradigm that aims to shrink the norms of backdoor-related neurons by incorporating sharpness-aware minimization with fine-tuning.
  arXiv  Detail & Related papers  (2023-04-24T05:13:52Z)
• Backdoor Defense via Deconfounded Representation Learning [17.28760299048368]
  We propose a Causality-inspired Backdoor Defense (CBD) to learn deconfounded representations for reliable classification. CBD is effective in reducing backdoor threats while maintaining high accuracy in predicting benign samples.
  arXiv  Detail & Related papers  (2023-03-13T02:25:59Z)
• Backdoor Defense via Suppressing Model Shortcuts [91.30995749139012]
  In this paper, we explore the backdoor mechanism from the angle of the model structure. We demonstrate that the attack success rate (ASR) decreases significantly when reducing the outputs of some key skip connections.
  arXiv  Detail & Related papers  (2022-11-02T15:39:19Z)
• Invisible Backdoor Attacks Using Data Poisoning in the Frequency Domain [8.64369418938889]
  We propose a generalized backdoor attack method based on the frequency domain. It can implement backdoor implantation without mislabeling and accessing the training process. We evaluate our approach in the no-label and clean-label cases on three datasets.
  arXiv  Detail & Related papers  (2022-07-09T07:05:53Z)
• Few-shot Backdoor Defense Using Shapley Estimation [123.56934991060788]
  We develop a new approach called Shapley Pruning to mitigate backdoor attacks on deep neural networks. ShapPruning identifies the few infected neurons (under 1% of all neurons) and manages to protect the model's structure and accuracy. Experiments demonstrate the effectiveness and robustness of our method against various attacks and tasks.
  arXiv  Detail & Related papers  (2021-12-30T02:27:03Z)
• Anti-Backdoor Learning: Training Clean Models on Poisoned Data [17.648453598314795]
  Backdoor attack has emerged as a major security threat to deep neural networks (DNNs) We introduce the concept of emphanti-backdoor learning, aiming to train emphclean models given backdoor-poisoned data. We empirically show that ABL-trained models on backdoor-poisoned data achieve the same performance as they were trained on purely clean data.
  arXiv  Detail & Related papers  (2021-10-22T03:30:48Z)
• Black-box Detection of Backdoor Attacks with Limited Information and Data [56.0735480850555]
  We propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models.
  arXiv  Detail & Related papers  (2021-03-24T12:06:40Z)
• Backdoor Learning: A Survey [75.59571756777342]
  Backdoor attack intends to embed hidden backdoor into deep neural networks (DNNs) Backdoor learning is an emerging and rapidly growing research area. This paper presents the first comprehensive survey of this realm.
  arXiv  Detail & Related papers  (2020-07-17T04:09:20Z)
• Defending against Backdoor Attack on Deep Neural Networks [98.45955746226106]
  We study the so-called textitbackdoor attack, which injects a backdoor trigger to a small portion of training data. Experiments show that our method could effectively decrease the attack success rate, and also hold a high classification accuracy for clean images.
  arXiv  Detail & Related papers  (2020-02-26T02:03:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.