Data Poisoning Won't Save You From Facial Recognition

• URL: http://arxiv.org/abs/2106.14851v1
• Date: Mon, 28 Jun 2021 17:06:19 GMT
• Title: Data Poisoning Won't Save You From Facial Recognition
• Authors: Evani Radiya-Dixit, Florian Tram\`er
• Abstract summary: Data poisoning has been proposed as a compelling defense against facial recognition models trained on Web-scraped pictures. We demonstrate that this strategy provides a false sense of security. We evaluate two systems for poisoning attacks against large-scale facial recognition.
• Score: 1.14219428942199
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data poisoning has been proposed as a compelling defense against facial recognition models trained on Web-scraped pictures. By perturbing the images they post online, users can fool models into misclassifying future (unperturbed) pictures. We demonstrate that this strategy provides a false sense of security, as it ignores an inherent asymmetry between the parties: users' pictures are perturbed once and for all before being published (at which point they are scraped) and must thereafter fool all future models -- including models trained adaptively against the users' past attacks, or models that use technologies discovered after the attack. We evaluate two systems for poisoning attacks against large-scale facial recognition, Fawkes (500,000+ downloads) and LowKey. We demonstrate how an "oblivious" model trainer can simply wait for future developments in computer vision to nullify the protection of pictures collected in the past. We further show that an adversary with black-box access to the attack can (i) train a robust model that resists the perturbations of collected pictures and (ii) detect poisoned pictures uploaded online. We caution that facial recognition poisoning will not admit an "arms race" between attackers and defenders. Once perturbed pictures are scraped, the attack cannot be changed so any future successful defense irrevocably undermines users' privacy.

Related papers

• Clean-image Backdoor Attacks [34.051173092777844]
  We propose clean-image backdoor attacks which uncover that backdoors can still be injected via a fraction of incorrect labels. In our attacks, the attacker first seeks a trigger feature to divide the training images into two parts. The backdoor will be finally implanted into the target model after it is trained on the poisoned data.
  arXiv  Detail & Related papers  (2024-03-22T07:47:13Z)
• Object-oriented backdoor attack against image captioning [40.5688859498834]
  Backdoor attack against image classification task has been widely studied and proven to be successful. In this paper, we explore backdoor attack towards image captioning models by poisoning training data. Our method proves the weakness of image captioning models to backdoor attack and we hope this work can raise the awareness of defending against backdoor attack in the image captioning field.
  arXiv  Detail & Related papers  (2024-01-05T01:52:13Z)
• Pick your Poison: Undetectability versus Robustness in Data Poisoning Attacks [33.82164201455115]
  Deep image classification models trained on vast amounts of web-scraped data are susceptible to data poisoning. Existing work considers an effective defense as one that either (i) restores a model's integrity through repair or (ii) detects an attack. We argue that this approach overlooks a crucial trade-off: Attackers can increase at the expense of detectability (over-poisoning) or decrease detectability at the cost of robustness (under-poisoning)
  arXiv  Detail & Related papers  (2023-05-07T15:58:06Z)
• Untargeted Backdoor Attack against Object Detection [69.63097724439886]
  We design a poison-only backdoor attack in an untargeted manner, based on task characteristics. We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns.
  arXiv  Detail & Related papers  (2022-11-02T17:05:45Z)
• Restricted Black-box Adversarial Attack Against DeepFake Face Swapping [70.82017781235535]
  We introduce a practical adversarial attack that does not require any queries to the facial image forgery model. Our method is built on a substitute model persuing for face reconstruction and then transfers adversarial examples from the substitute model directly to inaccessible black-box DeepFake models.
  arXiv  Detail & Related papers  (2022-04-26T14:36:06Z)
• On the Effectiveness of Adversarial Training against Backdoor Attacks [111.8963365326168]
  A backdoored model always predicts a target class in the presence of a predefined trigger pattern. In general, adversarial training is believed to defend against backdoor attacks. We propose a hybrid strategy which provides satisfactory robustness across different backdoor attacks.
  arXiv  Detail & Related papers  (2022-02-22T02:24:46Z)
• Black-box Detection of Backdoor Attacks with Limited Information and Data [56.0735480850555]
  We propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models.
  arXiv  Detail & Related papers  (2021-03-24T12:06:40Z)
• Oriole: Thwarting Privacy against Trustworthy Deep Learning Models [16.224149190291048]
  We present Oriole, a system that combines the advantages of data poisoning attacks and evasion attacks. Our proposed Oriole system is able to effectively interfere with the performance of the Fawkes system to achieve promising attacking results.
  arXiv  Detail & Related papers  (2021-02-23T05:33:55Z)
• Online Alternate Generator against Adversarial Attacks [144.45529828523408]
  Deep learning models are notoriously sensitive to adversarial examples which are synthesized by adding quasi-perceptible noises on real images. We propose a portable defense method, online alternate generator, which does not need to access or modify the parameters of the target networks. The proposed method works by online synthesizing another image from scratch for an input image, instead of removing or destroying adversarial noises.
  arXiv  Detail & Related papers  (2020-09-17T07:11:16Z)
• Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning [71.17774313301753]
  We explore the robustness of self-supervised learned high-level representations by using them in the defense against adversarial attacks. Experimental results on the ASVspoof 2019 dataset demonstrate that high-level representations extracted by Mockingjay can prevent the transferability of adversarial examples.
  arXiv  Detail & Related papers  (2020-06-05T03:03:06Z)
• Fawkes: Protecting Privacy against Unauthorized Deep Learning Models [34.04323550970413]
  Fawkes is a system that helps individuals inoculate their images against unauthorized facial recognition models. We experimentally demonstrate that Fawkes provides 95+% protection against user recognition. We achieve 100% success in experiments against today's state-of-the-art facial recognition services.
  arXiv  Detail & Related papers  (2020-02-19T18:00:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.