FRAME : Comprehensive Risk Assessment Framework for Adversarial Machine Learning Threats

• URL: http://arxiv.org/abs/2508.17405v1
• Date: Sun, 24 Aug 2025 15:20:26 GMT
• Title: FRAME : Comprehensive Risk Assessment Framework for Adversarial Machine Learning Threats
• Authors: Avishag Shapira, Simon Shigol, Asaf Shabtai,
• Abstract summary: We present FRAME, the first comprehensive framework for assessing AML risks across diverse ML-based systems.<n> FRAME includes a novel risk assessment method that quantifies AML risks by systematically evaluating three key dimensions.<n>We developed a comprehensive structured dataset of AML attacks enabling context-aware risk assessment.
• Score: 11.660800166163272
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The widespread adoption of machine learning (ML) systems increased attention to their security and emergence of adversarial machine learning (AML) techniques that exploit fundamental vulnerabilities in ML systems, creating an urgent need for comprehensive risk assessment for ML-based systems. While traditional risk assessment frameworks evaluate conventional cybersecurity risks, they lack ability to address unique challenges posed by AML threats. Existing AML threat evaluation approaches focus primarily on technical attack robustness, overlooking crucial real-world factors like deployment environments, system dependencies, and attack feasibility. Attempts at comprehensive AML risk assessment have been limited to domain-specific solutions, preventing application across diverse systems. Addressing these limitations, we present FRAME, the first comprehensive and automated framework for assessing AML risks across diverse ML-based systems. FRAME includes a novel risk assessment method that quantifies AML risks by systematically evaluating three key dimensions: target system's deployment environment, characteristics of diverse AML techniques, and empirical insights from prior research. FRAME incorporates a feasibility scoring mechanism and LLM-based customization for system-specific assessments. Additionally, we developed a comprehensive structured dataset of AML attacks enabling context-aware risk assessment. From an engineering application perspective, FRAME delivers actionable results designed for direct use by system owners with only technical knowledge of their systems, without expertise in AML. We validated it across six diverse real-world applications. Our evaluation demonstrated exceptional accuracy and strong alignment with analysis by AML experts. FRAME enables organizations to prioritize AML risks, supporting secure AI deployment in real-world environments.

Related papers

• Goal-Driven Risk Assessment for LLM-Powered Systems: A Healthcare Case Study [0.5801044612920815]
  We propose a structured, goal driven risk assessment approach that contextualizes the threats with detailed attack vectors, preconditions, and attack paths through the use of attack trees.<n>This study harmonizes the state-of-the-art attacks to LLMs with conventional ones and presents possible attack paths applicable to similar systems.
  arXiv  Detail & Related papers  (2026-03-04T01:49:48Z)
• An Empirical Study on the Security Vulnerabilities of GPTs [48.12756684275687]
  GPTs are one kind of customized AI agents based on OpenAI's large language models.<n>We present an empirical study on the security vulnerabilities of GPTs.
  arXiv  Detail & Related papers  (2025-11-28T13:30:25Z)
• SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models [50.34706204154244]
  Acquiring reasoning capabilities catastrophically degrades inherited safety alignment.<n>Certain scenarios suffer 25 times higher attack rates.<n>Despite tight reasoning-answer safety coupling, MLRMs demonstrate nascent self-correction.
  arXiv  Detail & Related papers  (2025-04-09T06:53:23Z)
• MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks [109.53357276796655]
  Multimodal large language models (MLLMs) equipped with Retrieval Augmented Generation (RAG)<n>RAG enhances MLLMs by grounding responses in query-relevant external knowledge.<n>This reliance poses a critical yet underexplored safety risk: knowledge poisoning attacks.<n>We propose MM-PoisonRAG, a novel knowledge poisoning attack framework with two attack strategies.
  arXiv  Detail & Related papers  (2025-02-25T04:23:59Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models [75.67623347512368]
  We propose toolns, a comprehensive framework designed for conducting safety evaluations of MLLMs. Our framework consists of a comprehensive harmful query dataset and an automated evaluation protocol. Based on our framework, we conducted large-scale experiments on 15 widely-used open-source MLLMs and 6 commercial MLLMs.
  arXiv  Detail & Related papers  (2024-10-24T17:14:40Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.<n>Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.<n>However, the deployment of these agents in physical environments presents significant safety challenges.<n>This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Mapping LLM Security Landscapes: A Comprehensive Stakeholder Risk Assessment Proposal [0.0]
  We propose a risk assessment process using tools like the risk rating methodology which is used for traditional systems. We conduct scenario analysis to identify potential threat agents and map the dependent system components against vulnerability factors. We also map threats against three key stakeholder groups.
  arXiv  Detail & Related papers  (2024-03-20T05:17:22Z)
• System Safety Engineering for Social and Ethical ML Risks: A Case Study [0.5249805590164902]
  Governments, industry, and academia have undertaken efforts to identify and mitigate harms in ML-driven systems. Existing approaches are largely disjointed, ad-hoc and of unknown effectiveness. We focus in particular on how this analysis can extend to identifying social and ethical risks and developing concrete design-level controls to mitigate them.
  arXiv  Detail & Related papers  (2022-11-08T22:58:58Z)
• The Risks of Machine Learning Systems [11.105884571838818]
  A system's overall risk is influenced by its direct and indirect effects. Existing frameworks for ML risk/impact assessment often address an abstract notion of risk or do not concretize this dependence. First-order risks stem from aspects of the ML system, while second-order risks stem from the consequences of first-order risks.
  arXiv  Detail & Related papers  (2022-04-21T02:42:10Z)
• A Framework for Evaluating the Cybersecurity Risk of Real World, Machine Learning Production Systems [41.470634460215564]
  We develop an extension to the MulVAL attack graph generation and analysis framework to incorporate cyberattacks on ML production systems. Using the proposed extension, security practitioners can apply attack graph analysis methods in environments that include ML components.
  arXiv  Detail & Related papers  (2021-07-05T05:58:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.