Towards Evaluating the Robustness of Neural Networks Learned by Transduction

• URL: http://arxiv.org/abs/2110.14735v1
• Date: Wed, 27 Oct 2021 19:39:50 GMT
• Title: Towards Evaluating the Robustness of Neural Networks Learned by Transduction
• Authors: Jiefeng Chen, Xi Wu, Yang Guo, Yingyu Liang, Somesh Jha
• Abstract summary: Greedy Model Space Attack (GMSA) is an attack framework that can serve as a new baseline for evaluating transductive-learning based defenses. We show that GMSA, even with weak instantiations, can break previous transductive-learning based defenses.
• Score: 44.189248766285345
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There has been emerging interest in using transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020; Wang et al., ArXiv 2021). Compared to traditional defenses, these defense mechanisms "dynamically learn" the model based on test-time input; and theoretically, attacking these defenses reduces to solving a bilevel optimization problem, which poses difficulty in crafting adaptive attacks. In this paper, we examine these defense mechanisms from a principled threat analysis perspective. We formulate and analyze threat models for transductive-learning based defenses, and point out important subtleties. We propose the principle of attacking model space for solving bilevel attack objectives, and present Greedy Model Space Attack (GMSA), an attack framework that can serve as a new baseline for evaluating transductive-learning based defenses. Through systematic evaluation, we show that GMSA, even with weak instantiations, can break previous transductive-learning based defenses, which were resilient to previous attacks, such as AutoAttack (Croce and Hein, ICML 2020). On the positive side, we report a somewhat surprising empirical result of "transductive adversarial training": Adversarially retraining the model using fresh randomness at the test time gives a significant increase in robustness against attacks we consider.

Related papers

• Taking off the Rose-Tinted Glasses: A Critical Look at Adversarial ML Through the Lens of Evasion Attacks [11.830908033835728]
  We argue that overly permissive attack and overly restrictive defensive threat models have hampered defense development in the ML domain. We analyze adversarial machine learning from a system security perspective rather than an AI perspective.
  arXiv  Detail & Related papers  (2024-10-15T21:33:23Z)
• Robust Image Classification: Defensive Strategies against FGSM and PGD Adversarial Attacks [0.0]
  Adversarial attacks pose significant threats to the robustness of deep learning models in image classification. This paper explores and refines defense mechanisms against these attacks to enhance the resilience of neural networks.
  arXiv  Detail & Related papers  (2024-08-20T02:00:02Z)
• Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey [114.17568992164303]
  Adrial attacks and defenses in machine learning and deep neural network have been gaining significant attention. This survey provides a comprehensive overview of the recent advancements in the field of adversarial attack and defense techniques. New avenues of attack are also explored, including search-based, decision-based, drop-based, and physical-world attacks.
  arXiv  Detail & Related papers  (2023-03-11T04:19:31Z)
• The Space of Adversarial Strategies [6.295859509997257]
  Adversarial examples, inputs designed to induce worst-case behavior in machine learning models, have been extensively studied over the past decade. We propose a systematic approach to characterize worst-case (i.e., optimal) adversaries.
  arXiv  Detail & Related papers  (2022-09-09T20:53:11Z)
• Can Adversarial Training Be Manipulated By Non-Robust Features? [64.73107315313251]
  Adversarial training, originally designed to resist test-time adversarial examples, has shown to be promising in mitigating training-time availability attacks. We identify a novel threat model named stability attacks, which aims to hinder robust availability by slightly perturbing the training data. Under this threat, we find that adversarial training using a conventional defense budget $epsilon$ provably fails to provide test robustness in a simple statistical setting.
  arXiv  Detail & Related papers  (2022-01-31T16:25:25Z)
• Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS [70.60975663021952]
  We study blackbox adversarial attacks on network classifiers. We argue that attacker-defender fixed points are themselves general-sum games with complex phase transitions. We show that a continual learning approach is required to study attacker-defender dynamics.
  arXiv  Detail & Related papers  (2021-11-23T23:42:16Z)
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness [53.094682754683255]
  We propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the in adversarial attacks parameterized by a recurrent neural network. We develop a model-agnostic training algorithm to improve the ability of the learned when attacking unseen defenses.
  arXiv  Detail & Related papers  (2021-10-13T13:54:24Z)
• Towards Adversarial Robustness via Transductive Learning [41.47295098415148]
  There has been emerging interest to use transductive learning for adversarial robustness. We first formalize and analyze modeling aspects of transductive robustness. We present new theoretical and empirical evidence in support of the utility of transductive learning.
  arXiv  Detail & Related papers  (2021-06-15T19:32:12Z)
• Adversarial vs behavioural-based defensive AI with joint, continual and active learning: automated evaluation of robustness to deception, poisoning and concept drift [62.997667081978825]
  Recent advancements in Artificial Intelligence (AI) have brought new capabilities to behavioural analysis (UEBA) for cyber-security. In this paper, we present a solution to effectively mitigate this attack by improving the detection process and efficiently leveraging human expertise.
  arXiv  Detail & Related papers  (2020-01-13T13:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.