Related papers: Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack

Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack

URL: http://arxiv.org/abs/2405.16134v2
Date: Thu, 30 May 2024 04:45:11 GMT
Title: Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack
Authors: Mingli Zhu, Siyuan Liang, Baoyuan Wu,
Abstract summary: We re-investigate the characteristics of backdoored models after defense. We find that the original backdoors still exist in defense models derived from existing post-training defense strategies. We empirically show that these dormant backdoors can be easily re-activated during inference.
Score: 32.74007523929888
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Deep neural networks face persistent challenges in defending against backdoor attacks, leading to an ongoing battle between attacks and defenses. While existing backdoor defense strategies have shown promising performance on reducing attack success rates, can we confidently claim that the backdoor threat has truly been eliminated from the model? To address it, we re-investigate the characteristics of the backdoored models after defense (denoted as defense models). Surprisingly, we find that the original backdoors still exist in defense models derived from existing post-training defense strategies, and the backdoor existence is measured by a novel metric called backdoor existence coefficient. It implies that the backdoors just lie dormant rather than being eliminated. To further verify this finding, we empirically show that these dormant backdoors can be easily re-activated during inference, by manipulating the original trigger with well-designed tiny perturbation using universal adversarial attack. More practically, we extend our backdoor reactivation to black-box scenario, where the defense model can only be queried by the adversary during inference, and develop two effective methods, i.e., query-based and transfer-based backdoor re-activation attacks. The effectiveness of the proposed methods are verified on both image classification and multimodal contrastive learning (i.e., CLIP) tasks. In conclusion, this work uncovers a critical vulnerability that has never been explored in existing defense strategies, emphasizing the urgency of designing more robust and advanced backdoor defense mechanisms in the future.