Towards Assuring EU AI Act Compliance and Adversarial Robustness of LLMs

• URL: http://arxiv.org/abs/2410.05306v1
• Date: Fri, 4 Oct 2024 18:38:49 GMT
• Title: Towards Assuring EU AI Act Compliance and Adversarial Robustness of LLMs
• Authors: Tomas Bueno Momcilovic, Beat Buesser, Giulio Zizzo, Mark Purcell, Dian Balta,
• Abstract summary: Large language models are prone to misuse and vulnerable to security threats. The European Union's Artificial Intelligence Act seeks to enforce AI robustness in certain contexts.
• Score: 1.368472250332885
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are prone to misuse and vulnerable to security threats, raising significant safety and security concerns. The European Union's Artificial Intelligence Act seeks to enforce AI robustness in certain contexts, but faces implementation challenges due to the lack of standards, complexity of LLMs and emerging security vulnerabilities. Our research introduces a framework using ontologies, assurance cases, and factsheets to support engineers and stakeholders in understanding and documenting AI system compliance and security regarding adversarial robustness. This approach aims to ensure that LLMs adhere to regulatory standards and are equipped to counter potential threats.

Related papers

• Integrating Cybersecurity Frameworks into IT Security: A Comprehensive Analysis of Threat Mitigation Strategies and Adaptive Technologies [0.0]
  The cybersecurity threat landscape is constantly actively making it imperative to develop sound frameworks to protect the IT structures. This paper aims to discuss the application of cybersecurity frameworks into the IT security with focus placed on the role of such frameworks in addressing the changing nature of cybersecurity threats. The discussion also singles out such technologies as Artificial Intelligence (AI) and Machine Learning (ML) as the core for real-time threat detection and response mechanisms.
  arXiv  Detail & Related papers  (2025-02-02T03:38:48Z)
• Large Language Model Safety: A Holistic Survey [35.42419096859496]
  The rapid development and deployment of large language models (LLMs) have introduced a new frontier in artificial intelligence. This survey provides a comprehensive overview of the current landscape of LLM safety, covering four major categories: value misalignment, robustness to adversarial attacks, misuse, and autonomous AI risks.
  arXiv  Detail & Related papers  (2024-12-23T16:11:27Z)
• Global Challenge for Safe and Secure LLMs Track 1 [57.08717321907755]
  The Global Challenge for Safe and Secure Large Language Models (LLMs) is a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) This paper introduces the Global Challenge for Safe and Secure Large Language Models (LLMs), a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) to foster the development of advanced defense mechanisms against automated jailbreaking attacks.
  arXiv  Detail & Related papers  (2024-11-21T08:20:31Z)
• Building Trust: Foundations of Security, Safety and Transparency in AI [0.23301643766310373]
  We review the current security and safety scenarios while highlighting challenges such as tracking issues, remediation, and the apparent absence of AI model lifecycle and ownership processes. This paper aims to provide some of the foundational pieces for more standardized security, safety, and transparency in the development and operation of AI models.
  arXiv  Detail & Related papers  (2024-11-19T06:55:57Z)
• Defining and Evaluating Physical Safety for Large Language Models [62.4971588282174]
  Large Language Models (LLMs) are increasingly used to control robotic systems such as drones. Their risks of causing physical threats and harm in real-world applications remain unexplored. We classify the physical safety risks of drones into four categories: (1) human-targeted threats, (2) object-targeted threats, (3) infrastructure attacks, and (4) regulatory violations.
  arXiv  Detail & Related papers  (2024-11-04T17:41:25Z)
• Developing Assurance Cases for Adversarial Robustness and Regulatory Compliance in LLMs [1.368472250332885]
  We develop an approach to developing assurance cases for adversarial robustness and regulatory compliance in large language models (LLMs) We propose a layered framework incorporating guardrails at various stages of deployment, aimed at mitigating these attacks and ensuring compliance with the EU AI Act. We illustrate our method with two exemplary assurance cases, highlighting how different contexts demand tailored strategies to ensure robust and compliant AI systems.
  arXiv  Detail & Related papers  (2024-10-04T18:14:29Z)
• SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering [56.92068213969036]
  Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions. Recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue. We propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns.
  arXiv  Detail & Related papers  (2024-08-21T10:01:34Z)
• Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Model [73.8765529028288]
  We introduce a novel safety alignment challenge called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety alignment. To empirically investigate this problem, we developed the SIUO, a cross-modality benchmark encompassing 9 critical safety domains, such as self-harm, illegal activities, and privacy violations. Our findings reveal substantial safety vulnerabilities in both closed- and open-source LVLMs, underscoring the inadequacy of current models to reliably interpret and respond to complex, real-world scenarios.
  arXiv  Detail & Related papers  (2024-06-21T16:14:15Z)
• AI Risk Management Should Incorporate Both Safety and Security [185.68738503122114]
  We argue that stakeholders in AI risk management should be aware of the nuances, synergies, and interplay between safety and security. We introduce a unified reference framework to clarify the differences and interplay between AI safety and AI security.
  arXiv  Detail & Related papers  (2024-05-29T21:00:47Z)
• Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices [4.927763944523323]
  Large language models (LLMs) have significantly transformed the landscape of Natural Language Processing (NLP) This research paper thoroughly investigates security and privacy concerns related to LLMs from five thematic perspectives. The paper recommends promising avenues for future research to enhance the security and risk management of LLMs.
  arXiv  Detail & Related papers  (2024-03-19T07:10:58Z)
• Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science [65.77763092833348]
  Intelligent agents powered by large language models (LLMs) have demonstrated substantial promise in autonomously conducting experiments and facilitating scientific discoveries across various disciplines. While their capabilities are promising, these agents also introduce novel vulnerabilities that demand careful consideration for safety. This paper conducts a thorough examination of vulnerabilities in LLM-based agents within scientific domains, shedding light on potential risks associated with their misuse and emphasizing the need for safety measures.
  arXiv  Detail & Related papers  (2024-02-06T18:54:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.