Security Challenges in Autonomous Systems Design

• URL: http://arxiv.org/abs/2312.00018v2
• Date: Mon, 4 Dec 2023 03:45:38 GMT
• Title: Security Challenges in Autonomous Systems Design
• Authors: Mohammad Hamad, Sebastian Steinhorst,
• Abstract summary: With the independence from human control, cybersecurity of such systems becomes even more critical. With the independence from human control, cybersecurity of such systems becomes even more critical. This paper thoroughly discusses the state of the art, identifies emerging security challenges and proposes research directions.
• Score: 1.864621482724548
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous systems are emerging in many application domains. With the recent advancements in artificial intelligence and machine learning, sensor technology, perception algorithms and robotics, scenarios previously requiring strong human involvement can be handled by autonomous systems. With the independence from human control, cybersecurity of such systems becomes even more critical as no human intervention in case of undesired behavior is possible. In this context, this paper discusses emerging security challenges in autonomous systems design which arise in many domains such as autonomous incident response, risk assessment, data availability, systems interaction, trustworthiness, updatability, access control, as well as the reliability and explainability of machine learning methods. In all these areas, this paper thoroughly discusses the state of the art, identifies emerging security challenges and proposes research directions to address these challenges for developing secure autonomous systems.

Related papers

• Generative AI Agents in Autonomous Machines: A Safety Perspective [9.02400798202199]
  generative AI agents provide unparalleled capabilities, but they also have unique safety concerns. This work investigates the evolving safety requirements when generative models are integrated as agents into physical autonomous machines. We recommend the development and implementation of comprehensive safety scorecards for the use of generative AI technologies in autonomous machines.
  arXiv  Detail & Related papers  (2024-10-20T20:07:08Z)
• Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
  This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs. We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:42:21Z)
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
  We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
  arXiv  Detail & Related papers  (2024-05-10T17:38:32Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• Inherent Diverse Redundant Safety Mechanisms for AI-based Software Elements in Automotive Applications [1.6495054381576084]
  This paper explores the role and challenges of Artificial Intelligence (AI) algorithms in autonomous driving systems. A primary concern relates to the ability (and necessity) of AI models to generalize beyond their initial training data. This paper investigates the risk associated with overconfident AI models in safety-critical applications like autonomous driving.
  arXiv  Detail & Related papers  (2024-02-13T04:15:26Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• Security Considerations in AI-Robotics: A Survey of Current Methods, Challenges, and Opportunities [4.466887678364242]
  Motivated by the need to address the security concerns in AI-Robotics systems, this paper presents a comprehensive survey and taxonomy across three dimensions. We begin by surveying potential attack surfaces and provide mitigating defensive strategies. We then delve into ethical issues, such as dependency and psychological impact, as well as the legal concerns regarding accountability for these systems.
  arXiv  Detail & Related papers  (2023-10-12T17:54:20Z)
• System Safety and Artificial Intelligence [0.0]
  New applications of AI across societal domains come with new hazards. The field of system safety has dealt with accidents and harm in safety-critical systems. This chapter honors system safety pioneer Nancy Leveson.
  arXiv  Detail & Related papers  (2022-02-18T16:37:54Z)
• Systems Challenges for Trustworthy Embodied Systems [0.0]
  A new generation of increasingly autonomous and self-learning systems, which we call embodied systems, is about to be developed. It is crucial to coordinate the behavior of embodied systems in a beneficial manner, ensure their compatibility with our human-centered social values, and design verifiably safe and reliable human-machine interaction. We are arguing that raditional systems engineering is coming to a climacteric from embedded to embodied systems, and with assuring the trustworthiness of dynamic federations of situationally aware, intent-driven, explorative, ever-evolving, largely non-predictable, and increasingly autonomous embodied systems in
  arXiv  Detail & Related papers  (2022-01-10T15:52:17Z)
• From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence [113.06484656032978]
  Article argues that embodied intelligence is a key driver for the advancement of machine learning technology. We highlight challenges and opportunities specific to embodied intelligence. We propose research directions which may significantly advance the state-of-the-art in robot learning.
  arXiv  Detail & Related papers  (2021-10-28T16:04:01Z)
• Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
  Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance. We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
  arXiv  Detail & Related papers  (2020-10-19T13:09:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.