Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies

• URL: http://arxiv.org/abs/2112.11471v1
• Date: Tue, 21 Dec 2021 19:00:02 GMT
• Title: Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies
• Authors: Vivian Lai, Chacha Chen, Q. Vera Liao, Alison Smith-Renner, Chenhao Tan
• Abstract summary: There is growing interest in the research community to augment human decision making with AI assistance. The emerging field of human-AI decision making must embrace empirical approaches to form a foundational understanding of how humans interact and work with AI.
• Score: 22.214709837295906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As AI systems demonstrate increasingly strong predictive performance, their adoption has grown in numerous domains. However, in high-stakes domains such as criminal justice and healthcare, full automation is often not desirable due to safety, ethical, and legal concerns, yet fully manual approaches can be inaccurate and time consuming. As a result, there is growing interest in the research community to augment human decision making with AI assistance. Besides developing AI technologies for this purpose, the emerging field of human-AI decision making must embrace empirical approaches to form a foundational understanding of how humans interact and work with AI to make decisions. To invite and help structure research efforts towards a science of understanding and improving human-AI decision making, we survey recent literature of empirical human-subject studies on this topic. We summarize the study design choices made in over 100 papers in three important aspects: (1) decision tasks, (2) AI models and AI assistance elements, and (3) evaluation metrics. For each aspect, we summarize current trends, discuss gaps in current practices of the field, and make a list of recommendations for future research. Our survey highlights the need to develop common frameworks to account for the design and research spaces of human-AI decision making, so that researchers can make rigorous choices in study design, and the research community can build on each other's work and produce generalizable scientific knowledge. We also hope this survey will serve as a bridge for HCI and AI communities to work together to mutually shape the empirical science and computational technologies for human-AI decision making.

Related papers

• AI for social science and social science of AI: A Survey [47.5235291525383]
  Recent advancements in artificial intelligence have sparked a rethinking of artificial general intelligence possibilities. The increasing human-like capabilities of AI are also attracting attention in social science research.
  arXiv  Detail & Related papers  (2024-01-22T10:57:09Z)
• On the Opportunities of Green Computing: A Survey [80.21955522431168]
  Artificial Intelligence (AI) has achieved significant advancements in technology and research with the development over several decades. The needs for high computing power brings higher carbon emission and undermines research fairness. To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic.
  arXiv  Detail & Related papers  (2023-11-01T11:16:41Z)
• Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge [22.21959942886099]
  We introduce a process-oriented notion of appropriate reliance called critical use that centers the human's ability to situate AI predictions against knowledge that is uniquely available to them but unavailable to the AI model. We conduct a randomized online experiment in a complex social decision-making setting: child maltreatment screening. We find that, by providing participants with accelerated, low-stakes opportunities to practice AI-assisted decision-making, novices came to exhibit patterns of disagreement with AI that resemble those of experienced workers.
  arXiv  Detail & Related papers  (2023-08-30T01:54:31Z)
• AI empowering research: 10 ways how science can benefit from AI [0.0]
  This article explores the transformative impact of artificial intelligence (AI) on scientific research. It highlights ten ways in which AI is revolutionizing the work of scientists, including powerful referencing tools, improved understanding of research problems, enhanced research question generation, optimized research design, stub data generation, data transformation, advanced data analysis, and AI-assisted reporting.
  arXiv  Detail & Related papers  (2023-07-17T18:41:18Z)
• Human-AI Coevolution [48.74579595505374]
  Coevolution AI is a process in which humans and AI algorithms continuously influence each other. This paper introduces Coevolution AI as the cornerstone for a new field of study at the intersection between AI and complexity science.
  arXiv  Detail & Related papers  (2023-06-23T18:10:54Z)
• Human-Centered Responsible Artificial Intelligence: Current & Future Trends [76.94037394832931]
  In recent years, the CHI community has seen significant growth in research on Human-Centered Responsible Artificial Intelligence. All of this work is aimed at developing AI that benefits humanity while being grounded in human rights and ethics, and reducing the potential harms of AI. In this special interest group, we aim to bring together researchers from academia and industry interested in these topics to map current and future research trends.
  arXiv  Detail & Related papers  (2023-02-16T08:59:42Z)
• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations [18.971689499890363]
  We identify and analyze 97core papers with human-based XAI evaluations over the past five years. Our research shows that XAI is spreading more rapidly in certain application domains, such as recommender systems. We propose practical guidelines on designing and conducting user studies for XAI researchers and practitioners.
  arXiv  Detail & Related papers  (2022-10-20T20:53:00Z)
• Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir" [76.44130385507894]
  This paper aims to ground what we dub a 'participatory turn' in AI design by synthesizing existing literature on participation and through empirical analysis of its current practices. Based on our literature synthesis and empirical research, this paper presents a conceptual framework for analyzing participatory approaches to AI design.
  arXiv  Detail & Related papers  (2021-11-01T17:57:04Z)
• Empowering Local Communities Using Artificial Intelligence [70.17085406202368]
  It has become an important topic to explore the impact of AI on society from a people-centered perspective. Previous works in citizen science have identified methods of using AI to engage the public in research. This article discusses the challenges of applying AI in Community Citizen Science.
  arXiv  Detail & Related papers  (2021-10-05T12:51:11Z)
• AI in society and culture: decision making and values [0.0]
  Several academic debates, social consultations and impact studies are available to reveal the key aspects of the changing human-machine ecosystem. In details, sociocultural filters, taxonomy of human-machine decisions and perspectives of value-based AI are in the focus of this literature review. For better understanding, it is proposed to invite stakeholders in the prepared large-scale survey about the next generation AI that investigates issues that go beyond the technology.
  arXiv  Detail & Related papers  (2020-04-29T07:09:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.