Detecting Backdoors in Neural Networks Using Novel Feature-Based Anomaly Detection

• URL: http://arxiv.org/abs/2011.02526v1
• Date: Wed, 4 Nov 2020 20:33:51 GMT
• Title: Detecting Backdoors in Neural Networks Using Novel Feature-Based Anomaly Detection
• Authors: Hao Fu, Akshaj Kumar Veldanda, Prashanth Krishnamurthy, Siddharth Garg, and Farshad Khorrami
• Abstract summary: This paper proposes a new defense against neural network backdooring attacks. It is based on the intuition that the feature extraction layers of a backdoored network embed new features to detect the presence of a trigger. To detect backdoors, the proposed defense uses two synergistic anomaly detectors trained on clean validation data.
• Score: 16.010654200489913
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a new defense against neural network backdooring attacks that are maliciously trained to mispredict in the presence of attacker-chosen triggers. Our defense is based on the intuition that the feature extraction layers of a backdoored network embed new features to detect the presence of a trigger and the subsequent classification layers learn to mispredict when triggers are detected. Therefore, to detect backdoors, the proposed defense uses two synergistic anomaly detectors trained on clean validation data: the first is a novelty detector that checks for anomalous features, while the second detects anomalous mappings from features to outputs by comparing with a separate classifier trained on validation data. The approach is evaluated on a wide range of backdoored networks (with multiple variations of triggers) that successfully evade state-of-the-art defenses. Additionally, we evaluate the robustness of our approach on imperceptible perturbations, scalability on large-scale datasets, and effectiveness under domain shift. This paper also shows that the defense can be further improved using data augmentation.

Related papers

• Twin Trigger Generative Networks for Backdoor Attacks against Object Detection [14.578800906364414]
  Object detectors, which are widely used in real-world applications, are vulnerable to backdoor attacks. Most research on backdoor attacks has focused on image classification, with limited investigation into object detection. We propose novel twin trigger generative networks to generate invisible triggers for implanting backdoors into models during training, and visible triggers for steady activation during inference.
  arXiv  Detail & Related papers  (2024-11-23T03:46:45Z)
• A Robust Likelihood Model for Novelty Detection [8.766411351797883]
  Current approaches to novelty or anomaly detection are based on deep neural networks. We propose a new prior that aims at learning a robust likelihood for the novelty test, as a defense against attacks. We also integrate the same prior with a state-of-the-art novelty detection approach.
  arXiv  Detail & Related papers  (2023-06-06T01:02:31Z)
• FreeEagle: Detecting Complex Neural Trojans in Data-Free Cases [50.065022493142116]
  Trojan attack on deep neural networks, also known as backdoor attack, is a typical threat to artificial intelligence. FreeEagle is the first data-free backdoor detection method that can effectively detect complex backdoor attacks.
  arXiv  Detail & Related papers  (2023-02-28T11:31:29Z)
• Untargeted Backdoor Attack against Object Detection [69.63097724439886]
  We design a poison-only backdoor attack in an untargeted manner, based on task characteristics. We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns.
  arXiv  Detail & Related papers  (2022-11-02T17:05:45Z)
• An anomaly detection approach for backdoored neural networks: face recognition as a case study [77.92020418343022]
  We propose a novel backdoored network detection method based on the principle of anomaly detection. We test our method on a novel dataset of backdoored networks and report detectability results with perfect scores.
  arXiv  Detail & Related papers  (2022-08-22T12:14:13Z)
• VPN: Verification of Poisoning in Neural Networks [11.221552724154988]
  We study another neural network security issue, namely data poisoning. In this case an attacker inserts a trigger into a subset of the training data, in such a way that at test time, this trigger in an input causes the trained model to misclassify to some target class. We show how to formulate the check for data poisoning as a property that can be checked with off-the-shelf verification tools.
  arXiv  Detail & Related papers  (2022-05-08T15:16:05Z)
• Detection and Continual Learning of Novel Face Presentation Attacks [23.13064343026656]
  State-of-the-art face antispoofing systems are still vulnerable to novel types of attacks that are never seen during training. In this paper, we enable a deep neural network to detect anomalies in the observed input data points as potential new types of attacks. We then use experience replay to update the model to incorporate knowledge about new types of attacks without forgetting the past learned attack types.
  arXiv  Detail & Related papers  (2021-08-27T01:33:52Z)
• Adversarially Robust One-class Novelty Detection [83.1570537254877]
  We show that existing novelty detectors are susceptible to adversarial examples. We propose a defense strategy that manipulates the latent space of novelty detectors to improve the robustness against adversarial examples.
  arXiv  Detail & Related papers  (2021-08-25T10:41:29Z)
• No Need to Know Physics: Resilience of Process-based Model-free Anomaly Detection for Industrial Control Systems [95.54151664013011]
  We present a novel framework to generate adversarial spoofing signals that violate physical properties of the system. We analyze four anomaly detectors published at top security conferences.
  arXiv  Detail & Related papers  (2020-12-07T11:02:44Z)
• Detection as Regression: Certified Object Detection by Median Smoothing [50.89591634725045]
  This work is motivated by recent progress on certified classification by randomized smoothing. We obtain the first model-agnostic, training-free, and certified defense for object detection against $ell$-bounded attacks.
  arXiv  Detail & Related papers  (2020-07-07T18:40:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.