Conflicting Interactions Among Protections Mechanisms for Machine Learning Models

• URL: http://arxiv.org/abs/2207.01991v1
• Date: Tue, 5 Jul 2022 12:18:06 GMT
• Title: Conflicting Interactions Among Protections Mechanisms for Machine Learning Models
• Authors: Sebastian Szyller, N. Asokan
• Abstract summary: ML models have become targets for various attacks. Research at the intersection of security and privacy, and ML has flourished. A solution that is optimal for a specific concern may interact negatively with solutions intended to address other concerns.
• Score: 15.047412609389983
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nowadays, systems based on machine learning (ML) are widely used in different domains. Given their popularity, ML models have become targets for various attacks. As a result, research at the intersection of security and privacy, and ML has flourished. The research community has been exploring the attack vectors and potential mitigations separately. However, practitioners will likely need to deploy defences against several threats simultaneously. A solution that is optimal for a specific concern may interact negatively with solutions intended to address other concerns. In this work, we explore the potential for conflicting interactions between different solutions that enhance the security/privacy of ML-base systems. We focus on model and data ownership; exploring how ownership verification techniques interact with other ML security/privacy techniques like differentially private training, and robustness against model evasion. We provide a framework, and conduct systematic analysis of pairwise interactions. We show that many pairs are incompatible. Where possible, we provide relaxations to the hyperparameters or the techniques themselves that allow for the simultaneous deployment. Lastly, we discuss the implications and provide guidelines for future work.

Related papers

• HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI Interactions [76.42274173122328]
  We present HAICOSYSTEM, a framework examining AI agent safety within diverse and complex social interactions. We run 1840 simulations based on 92 scenarios across seven domains (e.g., healthcare, finance, education) Our experiments show that state-of-the-art LLMs, both proprietary and open-sourced, exhibit safety risks in over 50% cases.
  arXiv  Detail & Related papers  (2024-09-24T19:47:21Z)
• How to Train your Antivirus: RL-based Hardening through the Problem-Space [22.056941223966255]
  Adversarial training, the sole defensive technique that can confer empirical robustness, is not applicable out of the box in this domain. We introduce a novel Reinforcement Learning approach for constructing adversarial examples, a constituent part of adversarially training a model against evasion.
  arXiv  Detail & Related papers  (2024-02-29T10:38:56Z)
• Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective [69.25513235556635]
  Adversarial machine learning (AML) studies the adversarial phenomenon of machine learning, which may make inconsistent or unexpected predictions with humans. Some paradigms have been recently developed to explore this adversarial phenomenon occurring at different stages of a machine learning system. We propose a unified mathematical framework to covering existing attack paradigms.
  arXiv  Detail & Related papers  (2023-02-19T02:12:21Z)
• "Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice [10.814642396601139]
  Motivated by the apparent gap between researchers and practitioners, this paper aims to bridge the two domains. We first present three real-world case studies from which we can glean practical insights unknown or neglected in research. Next we analyze all adversarial ML papers recently published in top security conferences, highlighting positive trends and blind spots.
  arXiv  Detail & Related papers  (2022-12-29T14:11:07Z)
• Learned Systems Security [30.39158287782567]
  A learned system uses machine learning (ML) internally to improve performance. We can expect such systems to be vulnerable to some adversarial-ML attacks. We develop a framework for identifying vulnerabilities that stem from the use of ML.
  arXiv  Detail & Related papers  (2022-12-20T15:09:30Z)
• RelaxLoss: Defending Membership Inference Attacks without Losing Utility [68.48117818874155]
  We propose a novel training framework based on a relaxed loss with a more achievable learning target. RelaxLoss is applicable to any classification model with added benefits of easy implementation and negligible overhead. Our approach consistently outperforms state-of-the-art defense mechanisms in terms of resilience against MIAs.
  arXiv  Detail & Related papers  (2022-07-12T19:34:47Z)
• A Framework for Understanding Model Extraction Attack and Defense [48.421636548746704]
  We study tradeoffs between model utility from a benign user's view and privacy from an adversary's view. We develop new metrics to quantify such tradeoffs, analyze their theoretical properties, and develop an optimization problem to understand the optimal adversarial attack and defense strategies.
  arXiv  Detail & Related papers  (2022-06-23T05:24:52Z)
• Multi-concept adversarial attacks [13.538643599990785]
  Test time attacks targeting a single ML model often neglect their impact on other ML models. We develop novel attack techniques that can simultaneously attack one set of ML models while preserving the accuracy of the other.
  arXiv  Detail & Related papers  (2021-10-19T22:14:19Z)
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety [54.478842696269304]
  The use of deep neural networks (DNNs) in safety-critical applications is challenging due to numerous model-inherent shortcomings. In recent years, a zoo of state-of-the-art techniques aiming to address these safety concerns has emerged. Our paper addresses both machine learning experts and safety engineers.
  arXiv  Detail & Related papers  (2021-04-29T09:54:54Z)
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models [64.03398193325572]
  Inference attacks against Machine Learning (ML) models allow adversaries to learn about training data, model parameters, etc. We concentrate on four attacks - namely, membership inference, model inversion, attribute inference, and model stealing. Our analysis relies on a modular re-usable software, ML-Doctor, which enables ML model owners to assess the risks of deploying their models.
  arXiv  Detail & Related papers  (2021-02-04T11:35:13Z)
• Adversarial Machine Learning: Bayesian Perspectives [0.4915744683251149]
  Adversarial Machine Learning (AML) is emerging as a major field aimed at protecting machine learning (ML) systems against security threats. In certain scenarios there may be adversaries that actively manipulate input data to fool learning systems. This creates a new class of security vulnerabilities that ML systems may face, and a new desirable property called adversarial robustness essential to trust operations.
  arXiv  Detail & Related papers  (2020-03-07T10:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.