On Large Language Models in Mission-Critical IT Governance: Are We Ready Yet?

• URL: http://arxiv.org/abs/2412.11698v2
• Date: Fri, 10 Jan 2025 13:35:37 GMT
• Title: On Large Language Models in Mission-Critical IT Governance: Are We Ready Yet?
• Authors: Matteo Esposito, Francesco Palagiano, Valentina Lenarduzzi, Davide Taibi,
• Abstract summary: Security of critical infrastructure has been a pressing concern since the advent of computers. Recent events reveal the increasing difficulty of meeting these challenges. We aim to explore practitioners' views on integrating Generative AI into the governance of IT MCSs.
• Score: 7.098487130130114
• License:
• Abstract: Context. The security of critical infrastructure has been a pressing concern since the advent of computers and has become even more critical in today's era of cyber warfare. Protecting mission-critical systems (MCSs), essential for national security, requires swift and robust governance, yet recent events reveal the increasing difficulty of meeting these challenges. Aim. Building on prior research showcasing the potential of Generative AI (GAI), such as Large Language Models, in enhancing risk analysis, we aim to explore practitioners' views on integrating GAI into the governance of IT MCSs. Our goal is to provide actionable insights and recommendations for stakeholders, including researchers, practitioners, and policymakers. Method. We designed a survey to collect practical experiences, concerns, and expectations of practitioners who develop and implement security solutions in the context of MCSs. Conclusions and Future Works. Our findings highlight that the safe use of LLMs in MCS governance requires interdisciplinary collaboration. Researchers should focus on designing regulation-oriented models and focus on accountability; practitioners emphasize data protection and transparency, while policymakers must establish a unified AI framework with global benchmarks to ensure ethical and secure LLMs-based MCS governance.

Related papers

• Safety at Scale: A Comprehensive Survey of Large Model Safety [299.801463557549]
  We present a comprehensive taxonomy of safety threats to large models, including adversarial attacks, data poisoning, backdoor attacks, jailbreak and prompt injection attacks, energy-latency attacks, data and model extraction attacks, and emerging agent-specific threats. We identify and discuss the open challenges in large model safety, emphasizing the need for comprehensive safety evaluations, scalable and effective defense mechanisms, and sustainable data practices.
  arXiv  Detail & Related papers  (2025-02-02T05:14:22Z)
• Integrating Cybersecurity Frameworks into IT Security: A Comprehensive Analysis of Threat Mitigation Strategies and Adaptive Technologies [0.0]
  The cybersecurity threat landscape is constantly actively making it imperative to develop sound frameworks to protect the IT structures. This paper aims to discuss the application of cybersecurity frameworks into the IT security with focus placed on the role of such frameworks in addressing the changing nature of cybersecurity threats. The discussion also singles out such technologies as Artificial Intelligence (AI) and Machine Learning (ML) as the core for real-time threat detection and response mechanisms.
  arXiv  Detail & Related papers  (2025-02-02T03:38:48Z)
• Large Language Model Safety: A Holistic Survey [35.42419096859496]
  The rapid development and deployment of large language models (LLMs) have introduced a new frontier in artificial intelligence. This survey provides a comprehensive overview of the current landscape of LLM safety, covering four major categories: value misalignment, robustness to adversarial attacks, misuse, and autonomous AI risks.
  arXiv  Detail & Related papers  (2024-12-23T16:11:27Z)
• Transparency, Security, and Workplace Training & Awareness in the Age of Generative AI [0.0]
  As AI technologies advance, ethical considerations, transparency, data privacy, and their impact on human labor intersect with the drive for innovation and efficiency. Our research explores publicly accessible large language models (LLMs) that often operate on the periphery, away from mainstream scrutiny. Specifically, we examine Gab AI, a platform that centers around unrestricted communication and privacy, allowing users to interact freely without censorship.
  arXiv  Detail & Related papers  (2024-12-19T17:40:58Z)
• Global Challenge for Safe and Secure LLMs Track 1 [57.08717321907755]
  The Global Challenge for Safe and Secure Large Language Models (LLMs) is a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) This paper introduces the Global Challenge for Safe and Secure Large Language Models (LLMs), a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) to foster the development of advanced defense mechanisms against automated jailbreaking attacks.
  arXiv  Detail & Related papers  (2024-11-21T08:20:31Z)
• SoK: Unifying Cybersecurity and Cybersafety of Multimodal Foundation Models with an Information Theory Approach [58.93030774141753]
  Multimodal foundation models (MFMs) represent a significant advancement in artificial intelligence. This paper conceptualizes cybersafety and cybersecurity in the context of multimodal learning. We present a comprehensive Systematization of Knowledge (SoK) to unify these concepts in MFMs, identifying key threats.
  arXiv  Detail & Related papers  (2024-11-17T23:06:20Z)
• World Models: The Safety Perspective [6.520366712367809]
  The concept of World Models (WM) has recently attracted a great deal of attention in the AI research community. We provide an in-depth analysis of state-of-the-art WMs and their impact in order to call on the research community to collaborate on improving the safety and trustworthiness of WM.
  arXiv  Detail & Related papers  (2024-11-12T10:15:11Z)
• Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trends [64.57762280003618]
  It is foreseeable that in the near future, LM-driven general AI agents will serve as essential tools in production tasks. This paper investigates scenarios involving the autonomous collaboration of future LM agents.
  arXiv  Detail & Related papers  (2024-09-22T14:09:49Z)
• Towards Trustworthy AI: A Review of Ethical and Robust Large Language Models [1.7466076090043157]
  Large Language Models (LLMs) could transform many fields, but their fast development creates significant challenges for oversight, ethical creation, and building user trust. This comprehensive review looks at key trust issues in LLMs, such as unintended harms, lack of transparency, vulnerability to attacks, alignment with human values, and environmental impact. To tackle these issues, we suggest combining ethical oversight, industry accountability, regulation, and public involvement.
  arXiv  Detail & Related papers  (2024-06-01T14:47:58Z)
• A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law [65.87885628115946]
  Large language models (LLMs) are revolutionizing the landscapes of finance, healthcare, and law. We highlight the instrumental role of LLMs in enhancing diagnostic and treatment methodologies in healthcare, innovating financial analytics, and refining legal interpretation and compliance strategies. We critically examine the ethics for LLM applications in these fields, pointing out the existing ethical concerns and the need for transparent, fair, and robust AI systems.
  arXiv  Detail & Related papers  (2024-05-02T22:43:02Z)
• Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science [65.77763092833348]
  Intelligent agents powered by large language models (LLMs) have demonstrated substantial promise in autonomously conducting experiments and facilitating scientific discoveries across various disciplines. While their capabilities are promising, these agents also introduce novel vulnerabilities that demand careful consideration for safety. This paper conducts a thorough examination of vulnerabilities in LLM-based agents within scientific domains, shedding light on potential risks associated with their misuse and emphasizing the need for safety measures.
  arXiv  Detail & Related papers  (2024-02-06T18:54:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.