Improving the State of the Art for Training Human-AI Teams: Technical Report #1 -- Results of Subject-Matter Expert Knowledge Elicitation Survey

• URL: http://arxiv.org/abs/2309.03211v1
• Date: Tue, 29 Aug 2023 13:42:52 GMT
• Title: Improving the State of the Art for Training Human-AI Teams: Technical Report #1 -- Results of Subject-Matter Expert Knowledge Elicitation Survey
• Authors: James E. McCarthy, Lillian Asiala, LeeAnn Maryeski, Nyla Warren
• Abstract summary: Sonalysts has begun an internal initiative to explore the training of human-AI teams. We decided to use Joint All-Domain Command and Control (JADC2) as a focus point. We engaged a number of Subject-Matter Experts (SMEs) with Command and Control experience to gain insight into developing a STE that embodied the teaming challenges associated with JADC2.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A consensus report produced for the Air Force Research Laboratory by the National Academies of Sciences, Engineering, and Mathematics documented a prevalent and increasing desire to support human-Artificial Intelligence (AI) teaming across military service branches. Sonalysts has begun an internal initiative to explore the training of human-AI teams. The first step in this effort is to develop a Synthetic Task Environment (STE) that is capable of facilitating research on human-AI teams. We decided to use Joint All-Domain Command and Control (JADC2) as a focus point for developing the STE because the volume of sensor inputs and decision options within the JADC2 concept likely requires the use of AI systems to enable timely decisions. Given this focus, we engaged a number of Subject-Matter Experts (SMEs) with Command and Control experience to gain insight into developing a STE that embodied the teaming challenges associated with JADC2. This report documents our initial engagement with those stakeholders. The research team identified thirteen Sonalysts employees with military backgrounds and Command and Control experience, and invited them to participate. Twelve respondents completed the survey. The team then analyzed the responses to identify themes that emerged and topics that would benefit from further analysis. The results indicated that our SMEs were amenable to research using tasks that were analogous to those encountered in military environments, as long as they required teams to process a great deal of incoming data to arrive at complex decisions. The SMEs felt that the testbed should support 'teams of teams" that represented a matrixed organization, and that it should support a robust array to spoken, text-based, and face-to-face communications.

Related papers

• Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI [129.08019405056262]
  Embodied Artificial Intelligence (Embodied AI) is crucial for achieving Artificial Intelligence (AGI) MLMs andWMs have attracted significant attention due to their remarkable perception, interaction, and reasoning capabilities. In this survey, we give a comprehensive exploration of the latest advancements in Embodied AI.
  arXiv  Detail & Related papers  (2024-07-09T14:14:47Z)
• Combining AI Control Systems and Human Decision Support via Robustness and Criticality [53.10194953873209]
  We extend a methodology for adversarial explanations (AE) to state-of-the-art reinforcement learning frameworks. We show that the learned AI control system demonstrates robustness against adversarial tampering. In a training / learning framework, this technology can improve both the AI's decisions and explanations through human interaction.
  arXiv  Detail & Related papers  (2024-07-03T15:38:57Z)
• A2C: A Modular Multi-stage Collaborative Decision Framework for Human-AI Teams [19.91751748232295]
  A2C is a multi-stage collaborative decision framework designed to enable robust decision-making within human-AI teams. It incorporates AI systems trained to recognise uncertainty in their decisions and defer to human experts when needed.
  arXiv  Detail & Related papers  (2024-01-25T02:31:52Z)
• Improving the State of the Art for Training Human-AI Teams: Technical Report #3 -- Analysis of Testbed Alternatives [0.0]
  Sonalysts is working on an initiative to expand its expertise in teaming to Human-Artificial Intelligence (AI) teams. To provide a foundation for that research, Sonalysts is investigating the development of a Synthetic Task Environment.
  arXiv  Detail & Related papers  (2023-08-29T14:06:30Z)
• Improving the State of the Art for Training Human-AI Teams: Technical Report #2 -- Results of Researcher Knowledge Elicitation Survey [0.0]
  Sonalysts has begun an internal initiative to explore the training of Human-AI teams. The first step in this effort is to develop a Synthetic Task Environment (STE) that is capable of facilitating research on Human-AI teams.
  arXiv  Detail & Related papers  (2023-08-29T13:54:32Z)
• Decision-Oriented Dialogue for Human-AI Collaboration [62.367222979251444]
  We describe a class of tasks called decision-oriented dialogues, in which AI assistants such as large language models (LMs) must collaborate with one or more humans via natural language to help them make complex decisions. We formalize three domains in which users face everyday decisions: (1) choosing an assignment of reviewers to conference papers, (2) planning a multi-step itinerary in a city, and (3) negotiating travel plans for a group of friends. For each task, we build a dialogue environment where agents receive a reward based on the quality of the final decision they reach.
  arXiv  Detail & Related papers  (2023-05-31T17:50:02Z)
• Onto4MAT: A Swarm Shepherding Ontology for Generalised Multi-Agent Teaming [2.9327503320877457]
  We provide a formal knowledge representation design that enables the swarm Artificial Intelligence to reason about its environment and system. We propose the Ontology for Generalised Multi-Agent Teaming, Onto4MAT, to enable more effective teaming between humans and teams.
  arXiv  Detail & Related papers  (2022-03-24T09:36:50Z)
• An Uncommon Task: Participatory Design in Legal AI [64.54460979588075]
  We examine a notable yet understudied AI design process in the legal domain that took place over a decade ago. We show how an interactive simulation methodology allowed computer scientists and lawyers to become co-designers.
  arXiv  Detail & Related papers  (2022-03-08T15:46:52Z)
• Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir" [76.44130385507894]
  This paper aims to ground what we dub a 'participatory turn' in AI design by synthesizing existing literature on participation and through empirical analysis of its current practices. Based on our literature synthesis and empirical research, this paper presents a conceptual framework for analyzing participatory approaches to AI design.
  arXiv  Detail & Related papers  (2021-11-01T17:57:04Z)
• Explaining decisions made with AI: A workbook (Use case 1: AI-assisted recruitment tool) [0.0]
  The Alan Turing Institute and the Information Commissioner's Office have been working together to tackle the difficult issues surrounding explainable AI. The ultimate product of this joint endeavour, Explaining decisions made with AI, published in May 2020, is the most comprehensive practical guidance on AI explanation produced anywhere to date. The goal of the workbook is to summarise some of main themes from Explaining decisions made with AI and then to provide the materials for a workshop exercise.
  arXiv  Detail & Related papers  (2021-03-20T17:03:50Z)
• Artificial Intelligence for IT Operations (AIOPS) Workshop White Paper [50.25428141435537]
  Artificial Intelligence for IT Operations (AIOps) is an emerging interdisciplinary field arising in the intersection between machine learning, big data, streaming analytics, and the management of IT operations. Main aim of the AIOPS workshop is to bring together researchers from both academia and industry to present their experiences, results, and work in progress in this field.
  arXiv  Detail & Related papers  (2021-01-15T10:43:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.