TanksWorld: A Multi-Agent Environment for AI Safety Research
- URL: http://arxiv.org/abs/2002.11174v1
- Date: Tue, 25 Feb 2020 21:00:52 GMT
- Title: TanksWorld: A Multi-Agent Environment for AI Safety Research
- Authors: Corban G. Rivera, Olivia Lyons, Arielle Summitt, Ayman Fatima, Ji Pak,
William Shao, Robert Chalmers, Aryeh Englander, Edward W. Staley, I-Jeng
Wang, Ashley J. Llorens
- Abstract summary: The ability to create artificial intelligence capable of performing complex tasks is rapidly outpacing our ability to ensure the safe and assured operation of AI-enabled systems.
Recent simulation environments to illustrate AI safety risks are relatively simple or narrowly-focused on a particular issue.
We introduce the AI safety TanksWorld as an environment for AI safety research with three essential aspects.
- Score: 5.218815947097599
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The ability to create artificial intelligence (AI) capable of performing
complex tasks is rapidly outpacing our ability to ensure the safe and assured
operation of AI-enabled systems. Fortunately, a landscape of AI safety research
is emerging in response to this asymmetry and yet there is a long way to go. In
particular, recent simulation environments created to illustrate AI safety
risks are relatively simple or narrowly-focused on a particular issue. Hence,
we see a critical need for AI safety research environments that abstract
essential aspects of complex real-world applications. In this work, we
introduce the AI safety TanksWorld as an environment for AI safety research
with three essential aspects: competing performance objectives, human-machine
teaming, and multi-agent competition. The AI safety TanksWorld aims to
accelerate the advancement of safe multi-agent decision-making algorithms by
providing a software framework to support competitions with both system
performance and safety objectives. As a work in progress, this paper introduces
our research objectives and learning environment with reference code and
baseline performance metrics to follow in a future work.
Related papers
- AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways [10.16690494897609]
An Artificial Intelligence (AI) agent is a software entity that autonomously performs tasks or makes decisions based on pre-defined objectives and data inputs.
This survey delves into the emerging security threats faced by AI agents, categorizing them into four critical knowledge gaps.
By systematically reviewing these threats, this paper highlights both the progress made and the existing limitations in safeguarding AI agents.
arXiv Detail & Related papers (2024-06-04T01:22:31Z) - Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs.
We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
arXiv Detail & Related papers (2024-05-14T09:42:21Z) - Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI.
The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees.
We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
arXiv Detail & Related papers (2024-05-10T17:38:32Z) - Quantifying AI Vulnerabilities: A Synthesis of Complexity, Dynamical Systems, and Game Theory [0.0]
We propose a novel approach that introduces three metrics: System Complexity Index (SCI), Lyapunov Exponent for AI Stability (LEAIS), and Nash Equilibrium Robustness (NER)
SCI quantifies the inherent complexity of an AI system, LEAIS captures its stability and sensitivity to perturbations, and NER evaluates its strategic robustness against adversarial manipulation.
arXiv Detail & Related papers (2024-04-07T07:05:59Z) - Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark [13.082034905010286]
We present an environment suite called Safety-Gymnasium, which encompasses safety-critical tasks in both single and multi-agent scenarios.
We offer a library of algorithms named Safe Policy Optimization (SafePO), comprising 16 state-of-the-art SafeRL algorithms.
arXiv Detail & Related papers (2023-10-19T08:19:28Z) - Cyber Security Requirements for Platforms Enhancing AI Reproducibility [0.0]
This study focuses on the field of artificial intelligence (AI) and introduces a new framework for evaluating AI platforms.
Five popular AI platforms; Floydhub, BEAT, Codalab, Kaggle, and OpenML were assessed.
The analysis revealed that none of these platforms fully incorporates the necessary cyber security measures.
arXiv Detail & Related papers (2023-09-27T09:43:46Z) - AI Maintenance: A Robustness Perspective [91.28724422822003]
We introduce highlighted robustness challenges in the AI lifecycle and motivate AI maintenance by making analogies to car maintenance.
We propose an AI model inspection framework to detect and mitigate robustness risks.
Our proposal for AI maintenance facilitates robustness assessment, status tracking, risk scanning, model hardening, and regulation throughout the AI lifecycle.
arXiv Detail & Related papers (2023-01-08T15:02:38Z) - Evaluating Model-free Reinforcement Learning toward Safety-critical
Tasks [70.76757529955577]
This paper revisits prior work in this scope from the perspective of state-wise safe RL.
We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection.
To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
arXiv Detail & Related papers (2022-12-12T06:30:17Z) - Proceedings of the Artificial Intelligence for Cyber Security (AICS)
Workshop at AAAI 2022 [55.573187938617636]
The workshop will focus on the application of AI to problems in cyber security.
Cyber systems generate large volumes of data, utilizing this effectively is beyond human capabilities.
arXiv Detail & Related papers (2022-02-28T18:27:41Z) - Proceedings of the Robust Artificial Intelligence System Assurance
(RAISA) Workshop 2022 [0.0]
The RAISA workshop will focus on research, development and application of robust artificial intelligence (AI) and machine learning (ML) systems.
Rather than studying robustness with respect to particular ML algorithms, our approach will be to explore robustness assurance at the system architecture level.
arXiv Detail & Related papers (2022-02-10T01:15:50Z) - Watch-And-Help: A Challenge for Social Perception and Human-AI
Collaboration [116.28433607265573]
We introduce Watch-And-Help (WAH), a challenge for testing social intelligence in AI agents.
In WAH, an AI agent needs to help a human-like agent perform a complex household task efficiently.
We build VirtualHome-Social, a multi-agent household environment, and provide a benchmark including both planning and learning based baselines.
arXiv Detail & Related papers (2020-10-19T21:48:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.