ADMIn: Attacks on Dataset, Model and Input. A Threat Model for AI Based Software

• URL: http://arxiv.org/abs/2401.07960v1
• Date: Mon, 15 Jan 2024 20:55:21 GMT
• Title: ADMIn: Attacks on Dataset, Model and Input. A Threat Model for AI Based Software
• Authors: Vimal Kumar, Juliette Mayo, Khadija Bahiss,
• Abstract summary: We present a threat model that can be used to uncover threats to AI based software. The threat model consists of two main parts, a model of the software development process for AI based software and an attack taxonomy. We apply the threat model to two real life AI based software and discuss the process and the threats found.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Machine learning (ML) and artificial intelligence (AI) techniques have now become commonplace in software products and services. When threat modelling a system, it is therefore important that we consider threats unique to ML and AI techniques, in addition to threats to our software. In this paper, we present a threat model that can be used to systematically uncover threats to AI based software. The threat model consists of two main parts, a model of the software development process for AI based software and an attack taxonomy that has been developed using attacks found in adversarial AI research. We apply the threat model to two real life AI based software and discuss the process and the threats found.

Related papers

• Threat Me Right: A Human HARMS Threat Model for Technical Systems [4.096453902709292]
  We discuss traditional threat modelling methods and their shortcomings. We propose a new threat modelling framework (HARMS) to identify non-technical and human factors harms.
  arXiv  Detail & Related papers  (2025-02-10T23:13:41Z)
• Is Generative AI the Next Tactical Cyber Weapon For Threat Actors? Unforeseen Implications of AI Generated Cyber Attacks [0.0]
  This paper delves into the escalating threat posed by the misuse of AI, specifically through the use of Large Language Models (LLMs) Through a series of controlled experiments, the paper demonstrates how these models can be manipulated to bypass ethical and privacy safeguards to effectively generate cyber attacks. We also introduce Occupy AI, a customized, finetuned LLM specifically engineered to automate and execute cyberattacks.
  arXiv  Detail & Related papers  (2024-08-23T02:56:13Z)
• Asset-centric Threat Modeling for AI-based Systems [7.696807063718328]
  This paper presents ThreatFinderAI, an approach and tool to model AI-related assets, threats, countermeasures, and quantify residual risks. To evaluate the practicality of the approach, participants were tasked to recreate a threat model developed by cybersecurity experts of an AI-based healthcare platform. Overall, the solution's usability was well-perceived and effectively supports threat identification and risk discussion.
  arXiv  Detail & Related papers  (2024-03-11T08:40:01Z)
• Autonomous Threat Hunting: A Future Paradigm for AI-Driven Threat Intelligence [0.0]
  Review explores the amalgamation of artificial intelligence (AI) and traditional threat intelligence methodologies. Examines the transformative influence of AI and machine learning on conventional threat intelligence practices. Case studies and evaluations highlight success stories and lessons learned by organizations adopting AI-driven threat intelligence.
  arXiv  Detail & Related papers  (2023-12-30T17:36:08Z)
• Towards more Practical Threat Models in Artificial Intelligence Security [66.67624011455423]
  Recent works have identified a gap between research and practice in artificial intelligence security. We revisit the threat models of the six most studied attacks in AI security research and match them to AI usage in practice.
  arXiv  Detail & Related papers  (2023-11-16T16:09:44Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• AI Maintenance: A Robustness Perspective [91.28724422822003]
  We introduce highlighted robustness challenges in the AI lifecycle and motivate AI maintenance by making analogies to car maintenance. We propose an AI model inspection framework to detect and mitigate robustness risks. Our proposal for AI maintenance facilitates robustness assessment, status tracking, risk scanning, model hardening, and regulation throughout the AI lifecycle.
  arXiv  Detail & Related papers  (2023-01-08T15:02:38Z)
• CTI4AI: Threat Intelligence Generation and Sharing after Red Teaming AI Models [0.0]
  A need to identify system vulnerabilities, potential threats, characterize properties that will enhance system robustness. A secondary need is to share this AI security threat intelligence between different stakeholders like, model developers, users, and AI/ML security professionals. In this paper, we create and describe a prototype system CTI4AI, to overcome the need to methodically identify and share AI/ML specific vulnerabilities and threat intelligence.
  arXiv  Detail & Related papers  (2022-08-16T00:16:58Z)
• Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
  We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering. In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)
• Proceedings of the Artificial Intelligence for Cyber Security (AICS) Workshop at AAAI 2022 [55.573187938617636]
  The workshop will focus on the application of AI to problems in cyber security. Cyber systems generate large volumes of data, utilizing this effectively is beyond human capabilities.
  arXiv  Detail & Related papers  (2022-02-28T18:27:41Z)
• The Threat of Offensive AI to Organizations [52.011307264694665]
  This survey explores the threat of offensive AI on organizations. First, we discuss how AI changes the adversary's methods, strategies, goals, and overall attack model. Then, through a literature review, we identify 33 offensive AI capabilities which adversaries can use to enhance their attacks.
  arXiv  Detail & Related papers  (2021-06-30T01:03:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.