Collaborative AI Needs Stronger Assurances Driven by Risks

• URL: http://arxiv.org/abs/2112.00740v1
• Date: Wed, 1 Dec 2021 15:24:21 GMT
• Title: Collaborative AI Needs Stronger Assurances Driven by Risks
• Authors: Jubril Gbolahan Adigun, Matteo Camilli, Michael Felderer, Andrea Giusti, Dominik T Matt, Anna Perini, Barbara Russo, Angelo Susi
• Abstract summary: Collaborative AI systems (CAISs) aim at working together with humans in a shared space to achieve a common goal. Building such systems with strong assurances of compliance with requirements, domain-specific standards and regulations is of greatest importance.
• Score: 5.657409854809804
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Collaborative AI systems (CAISs) aim at working together with humans in a shared space to achieve a common goal. This critical setting yields hazardous circumstances that could harm human beings. Thus, building such systems with strong assurances of compliance with requirements, domain-specific standards and regulations is of greatest importance. Only few scale impact has been reported so far for such systems since much work remains to manage possible risks. We identify emerging problems in this context and then we report our vision, as well as the progress of our multidisciplinary research team composed of software/systems, and mechatronics engineers to develop a risk-driven assurance process for CAISs.

Related papers

• HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI Interactions [76.42274173122328]
  We present HAICOSYSTEM, a framework examining AI agent safety within diverse and complex social interactions. We run 1840 simulations based on 92 scenarios across seven domains (e.g., healthcare, finance, education) Our experiments show that state-of-the-art LLMs, both proprietary and open-sourced, exhibit safety risks in over 50% cases.
  arXiv  Detail & Related papers  (2024-09-24T19:47:21Z)
• Safeguarding AI Agents: Developing and Analyzing Safety Architectures [0.0]
  This paper addresses the need for safety measures in AI systems that collaborate with human teams. We propose and evaluate three frameworks to enhance safety protocols in AI agent systems. We conclude that these frameworks can significantly strengthen the safety and security of AI agent systems.
  arXiv  Detail & Related papers  (2024-09-03T10:14:51Z)
• EAIRiskBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [47.69642609574771]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction. Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results. However, the deployment of these agents in physical environments presents significant safety challenges. This study introduces EAIRiskBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• Cross-Modality Safety Alignment [73.8765529028288]
  We introduce a novel safety alignment challenge called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety alignment. To empirically investigate this problem, we developed the SIUO, a cross-modality benchmark encompassing 9 critical safety domains, such as self-harm, illegal activities, and privacy violations. Our findings reveal substantial safety vulnerabilities in both closed- and open-source LVLMs, underscoring the inadequacy of current models to reliably interpret and respond to complex, real-world scenarios.
  arXiv  Detail & Related papers  (2024-06-21T16:14:15Z)
• AI Risk Management Should Incorporate Both Safety and Security [185.68738503122114]
  We argue that stakeholders in AI risk management should be aware of the nuances, synergies, and interplay between safety and security. We introduce a unified reference framework to clarify the differences and interplay between AI safety and AI security.
  arXiv  Detail & Related papers  (2024-05-29T21:00:47Z)
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
  We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
  arXiv  Detail & Related papers  (2024-05-10T17:38:32Z)
• Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science [65.77763092833348]
  Intelligent agents powered by large language models (LLMs) have demonstrated substantial promise in autonomously conducting experiments and facilitating scientific discoveries across various disciplines. While their capabilities are promising, these agents also introduce novel vulnerabilities that demand careful consideration for safety. This paper conducts a thorough examination of vulnerabilities in LLM-based agents within scientific domains, shedding light on potential risks associated with their misuse and emphasizing the need for safety measures.
  arXiv  Detail & Related papers  (2024-02-06T18:54:07Z)
• A Framework for Exploring the Consequences of AI-Mediated Enterprise Knowledge Access and Identifying Risks to Workers [3.4568218861862556]
  This paper presents the Consequence-Mechanism-Risk framework to identify risks to workers from AI-mediated enterprise knowledge access systems. We have drawn on wide-ranging literature detailing risks to workers, and categorised risks as being to worker value, power, and wellbeing. Future work could apply this framework to other technological systems to promote the protection of workers and other groups.
  arXiv  Detail & Related papers  (2023-12-08T17:05:40Z)
• A Brief Overview of AI Governance for Responsible Machine Learning Systems [3.222802562733787]
  This position paper seeks to present a brief introduction to AI governance, which is a framework designed to oversee the responsible use of AI. Due to the probabilistic nature of AI, the risks associated with it are far greater than traditional technologies.
  arXiv  Detail & Related papers  (2022-11-21T23:48:51Z)
• Towards Risk Modeling for Collaborative AI [5.941104748966331]
  Collaborative AI systems aim at working together with humans in a shared space to achieve a common goal. This setting imposes potentially hazardous circumstances due to contacts that could harm human beings. We introduce a risk modeling approach tailored to Collaborative AI systems.
  arXiv  Detail & Related papers  (2021-03-12T18:53:06Z)
• Towards a Policy-as-a-Service Framework to Enable Compliant, Trustworthy AI and HRI Systems in the Wild [7.225523345649149]
  Building trustworthy autonomous systems is challenging for many reasons beyond simply trying to engineer agents that 'always do the right thing' There is a broader context that is often not considered within AI and HRI: that the problem of trustworthiness is inherently socio-technical. This paper emphasizes the "fuzzy" socio-technical aspects of trustworthiness and the need for their careful consideration during both design and deployment.
  arXiv  Detail & Related papers  (2020-10-06T18:32:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.