Improving the State of the Art for Training Human-AI Teams: Technical Report #2 -- Results of Researcher Knowledge Elicitation Survey

• URL: http://arxiv.org/abs/2309.03212v1
• Date: Tue, 29 Aug 2023 13:54:32 GMT
• Title: Improving the State of the Art for Training Human-AI Teams: Technical Report #2 -- Results of Researcher Knowledge Elicitation Survey
• Authors: James E. McCarthy, Lillian Asiala, LeeAnn Maryeski, Dawn Sillars
• Abstract summary: Sonalysts has begun an internal initiative to explore the training of Human-AI teams. The first step in this effort is to develop a Synthetic Task Environment (STE) that is capable of facilitating research on Human-AI teams.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A consensus report produced for the Air Force Research Laboratory (AFRL) by the National Academies of Sciences, Engineering, and Mathematics documented a prevalent and increasing desire to support human-Artificial Intelligence (AI) teaming across military service branches. Sonalysts has begun an internal initiative to explore the training of Human-AI teams. The first step in this effort is to develop a Synthetic Task Environment (STE) that is capable of facilitating research on Human-AI teams. Our goal is to create a STE that offers a task environment that could support the breadth of research that stakeholders plan to perform within this domain. As a result, we wanted to sample the priorities of the relevant research community broadly, and the effort documented in this report is our initial attempt to do so. We created a survey that featured two types of questions. The first asked respondents to report their agreement with STE features that we anticipated might be important. The second represented open-ended questions that asked respondents to specify their priorities within several dimensions of the anticipated STE. The research team invited nineteen researchers from academic and Government labs to participate, and 11 were able to complete the survey. The team analyzed their responses to identify themes that emerged and topics that would benefit from further analysis. The most significant finding of the survey was that a number of researchers felt that various open-source STEs that would meet our needs already exist. Researchers also emphasized the need for automated transcription and coding tools to ease the burden of assessing inter-team communications; the importance of robust data capture and export capabilities; and the desirability of extensive flexibility across many aspects of the tool.

Related papers

• Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI [95.96983812740683]
  Embodied Artificial Intelligence (Embodied AI) is crucial for achieving Artificial Intelligence (AGI) MLMs andWMs have attracted significant attention due to their remarkable perception, interaction, and reasoning capabilities. In this survey, we give a comprehensive exploration of the latest advancements in Embodied AI.
  arXiv  Detail & Related papers  (2024-07-09T14:14:47Z)
• SurveyAgent: A Conversational System for Personalized and Efficient Research Survey [50.04283471107001]
  This paper introduces SurveyAgent, a novel conversational system designed to provide personalized and efficient research survey assistance to researchers. SurveyAgent integrates three key modules: Knowledge Management for organizing papers, Recommendation for discovering relevant literature, and Query Answering for engaging with content on a deeper level. Our evaluation demonstrates SurveyAgent's effectiveness in streamlining research activities, showcasing its capability to facilitate how researchers interact with scientific literature.
  arXiv  Detail & Related papers  (2024-04-09T15:01:51Z)
• Improving the State of the Art for Training Human-AI Teams: Technical Report #3 -- Analysis of Testbed Alternatives [0.0]
  Sonalysts is working on an initiative to expand its expertise in teaming to Human-Artificial Intelligence (AI) teams. To provide a foundation for that research, Sonalysts is investigating the development of a Synthetic Task Environment.
  arXiv  Detail & Related papers  (2023-08-29T14:06:30Z)
• Improving the State of the Art for Training Human-AI Teams: Technical Report #1 -- Results of Subject-Matter Expert Knowledge Elicitation Survey [0.0]
  Sonalysts has begun an internal initiative to explore the training of human-AI teams. We decided to use Joint All-Domain Command and Control (JADC2) as a focus point. We engaged a number of Subject-Matter Experts (SMEs) with Command and Control experience to gain insight into developing a STE that embodied the teaming challenges associated with JADC2.
  arXiv  Detail & Related papers  (2023-08-29T13:42:52Z)
• Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys [71.28049144033773]
  We propose a Survey of Surveys (SoS) for total technologies of autonomous driving (AD) and intelligent vehicles (IVs) This article is the first SoS with milestones in AD and IVs, which constitutes our complete research work together with two other technical surveys.
  arXiv  Detail & Related papers  (2023-03-30T08:31:22Z)
• Automated, not Automatic: Needs and Practices in European Fact-checking Organizations as a basis for Designing Human-centered AI Systems [0.7874708385247353]
  Despite existing research, there is still a gap between the fact-checking practitioners' needs and the current AI research. In this study, we conducted semi-structured in-depth interviews with Central European fact-checkers. The information behavior and requirements on desired supporting tools were analyzed using iterative bottom-up content analysis.
  arXiv  Detail & Related papers  (2022-11-22T10:18:09Z)
• Researching Alignment Research: Unsupervised Analysis [14.699455652461726]
  AI alignment research is dedicated to ensuring that artificial intelligence (AI) benefits humans. In this project, we collected and analyzed existing AI alignment research. We found that the field is growing quickly, with several subfields emerging in parallel.
  arXiv  Detail & Related papers  (2022-06-06T18:24:17Z)
• Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies [22.214709837295906]
  There is growing interest in the research community to augment human decision making with AI assistance. The emerging field of human-AI decision making must embrace empirical approaches to form a foundational understanding of how humans interact and work with AI.
  arXiv  Detail & Related papers  (2021-12-21T19:00:02Z)
• Wizard of Search Engine: Access to Information Through Conversations with Search Engines [58.53420685514819]
  We make efforts to facilitate research on CIS from three aspects. We formulate a pipeline for CIS with six sub-tasks: intent detection (ID), keyphrase extraction (KE), action prediction (AP), query selection (QS), passage selection (PS) and response generation (RG) We release a benchmark dataset, called wizard of search engine (WISE), which allows for comprehensive and in-depth research on all aspects of CIS.
  arXiv  Detail & Related papers  (2021-05-18T06:35:36Z)
• Artificial Intelligence for IT Operations (AIOPS) Workshop White Paper [50.25428141435537]
  Artificial Intelligence for IT Operations (AIOps) is an emerging interdisciplinary field arising in the intersection between machine learning, big data, streaming analytics, and the management of IT operations. Main aim of the AIOPS workshop is to bring together researchers from both academia and industry to present their experiences, results, and work in progress in this field.
  arXiv  Detail & Related papers  (2021-01-15T10:43:10Z)
• Conversations with Documents. An Exploration of Document-Centered Assistance [55.60379539074692]
  Document-centered assistance, for example, to help an individual quickly review a document, has seen less significant progress. We present a survey to understand the space of document-centered assistance and the capabilities people expect in this scenario. We present a set of initial machine learned models that show that (a) we can accurately detect document-centered questions, and (b) we can build reasonably accurate models for answering such questions.
  arXiv  Detail & Related papers  (2020-01-27T17:10:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.