Adversarial for Good? How the Adversarial ML Community's Values Impede Socially Beneficial Uses of Attacks

• URL: http://arxiv.org/abs/2107.10302v1
• Date: Sun, 11 Jul 2021 13:51:52 GMT
• Title: Adversarial for Good? How the Adversarial ML Community's Values Impede Socially Beneficial Uses of Attacks
• Authors: Kendra Albert, Maggie Delano, Bogdan Kulynych, Ram Shankar Siva Kumar
• Abstract summary: adversarial machine learning (ML) attacks have the potential to be used "for good" But most research on adversarial ML has not engaged in developing tools for resistance against ML systems.
• Score: 1.2664869982542892
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Attacks from adversarial machine learning (ML) have the potential to be used "for good": they can be used to run counter to the existing power structures within ML, creating breathing space for those who would otherwise be the targets of surveillance and control. But most research on adversarial ML has not engaged in developing tools for resistance against ML systems. Why? In this paper, we review the broader impact statements that adversarial ML researchers wrote as part of their NeurIPS 2020 papers and assess the assumptions that authors have about the goals of their work. We also collect information about how authors view their work's impact more generally. We find that most adversarial ML researchers at NeurIPS hold two fundamental assumptions that will make it difficult for them to consider socially beneficial uses of attacks: (1) it is desirable to make systems robust, independent of context, and (2) attackers of systems are normatively bad and defenders of systems are normatively good. That is, despite their expressed and supposed neutrality, most adversarial ML researchers believe that the goal of their work is to secure systems, making it difficult to conceptualize and build tools for disrupting the status quo.

Related papers

• ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming [64.86326523181553]
  ALERT is a large-scale benchmark to assess safety based on a novel fine-grained risk taxonomy. It aims to identify vulnerabilities, inform improvements, and enhance the overall safety of the language models.
  arXiv  Detail & Related papers  (2024-04-06T15:01:47Z)
• Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking [60.78524314357671]
  We investigate a novel category of jailbreak attacks specifically designed to target the cognitive structure and processes of large language models (LLMs) Our proposed cognitive overload is a black-box attack with no need for knowledge of model architecture or access to model weights. Experiments conducted on AdvBench and MasterKey reveal that various LLMs, including both popular open-source model Llama 2 and the proprietary model ChatGPT, can be compromised through cognitive overload.
  arXiv  Detail & Related papers  (2023-11-16T11:52:22Z)
• Adversarial Machine Learning for Social Good: Reframing the Adversary as an Ally [50.92232179802755]
  AdvML for Social Good (AdvML4G) is an emerging field that repurposes the AdvML bug to invent pro-social applications. This paper provides the first comprehensive review of the emerging field of AdvML4G.
  arXiv  Detail & Related papers  (2023-10-05T15:49:04Z)
• Vulnerability of Machine Learning Approaches Applied in IoT-based Smart Grid: A Review [51.31851488650698]
  Machine learning (ML) sees an increasing prevalence of being used in the internet-of-things (IoT)-based smart grid. adversarial distortion injected into the power signal will greatly affect the system's normal control and operation. It is imperative to conduct vulnerability assessment for MLsgAPPs applied in the context of safety-critical power systems.
  arXiv  Detail & Related papers  (2023-08-30T03:29:26Z)
• "Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice [10.814642396601139]
  Motivated by the apparent gap between researchers and practitioners, this paper aims to bridge the two domains. We first present three real-world case studies from which we can glean practical insights unknown or neglected in research. Next we analyze all adversarial ML papers recently published in top security conferences, highlighting positive trends and blind spots.
  arXiv  Detail & Related papers  (2022-12-29T14:11:07Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety [54.478842696269304]
  The use of deep neural networks (DNNs) in safety-critical applications is challenging due to numerous model-inherent shortcomings. In recent years, a zoo of state-of-the-art techniques aiming to address these safety concerns has emerged. Our paper addresses both machine learning experts and safety engineers.
  arXiv  Detail & Related papers  (2021-04-29T09:54:54Z)
• Legal Risks of Adversarial Machine Learning Research [0.7837881800517111]
  We show that studying or testing the security of any operational system potentially runs afoul the Computer Fraud and Abuse Act (CFAA) Our analysis show that because there is a split in how CFAA is interpreted, aspects of adversarial ML attacks may be sanctioned in some jurisdictions and not penalized in others. We argue that the court is likely to adopt a narrow construction of the CFAA, and that this will actually lead to better adversarial ML security outcomes in the long term.
  arXiv  Detail & Related papers  (2020-06-29T16:45:15Z)
• Adversarial Machine Learning: Bayesian Perspectives [0.4915744683251149]
  Adversarial Machine Learning (AML) is emerging as a major field aimed at protecting machine learning (ML) systems against security threats. In certain scenarios there may be adversaries that actively manipulate input data to fool learning systems. This creates a new class of security vulnerabilities that ML systems may face, and a new desirable property called adversarial robustness essential to trust operations.
  arXiv  Detail & Related papers  (2020-03-07T10:30:43Z)
• Politics of Adversarial Machine Learning [0.7837881800517111]
  adversarial machine-learning attacks and defenses have political dimensions. They enable or foreclose certain options for both the subjects of the machine learning systems and for those who deploy them. We show how defenses against adversarial attacks can be used to suppress dissent and limit attempts to investigate machine learning systems.
  arXiv  Detail & Related papers  (2020-02-01T01:15:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.