Tool or Tutor? Experimental evidence from AI deployment in cancer diagnosis

• URL: http://arxiv.org/abs/2502.16411v3
• Date: Sun, 30 Mar 2025 09:36:10 GMT
• Title: Tool or Tutor? Experimental evidence from AI deployment in cancer diagnosis
• Authors: Vivianna Fang He, Sihan Li, Phanish Puranam, Feng Lin,
• Abstract summary: We propose that AI-driven training and AI-assisted task completion can have a joint effect on human capability.<n>In a field experiment with 336 medical students, we manipulated AI deployment in training, in practice, and in both.
• Score: 3.0641365294595815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Professionals increasingly use Artificial Intelligence (AI) to enhance their capabilities and assist with task execution. While prior research has examined these uses separately, their potential interaction remains underexplored. We propose that AI-driven training ("tutor") and AI-assisted task completion ("tool") can have a joint effect on human capability and test this hypothesis in the context of lung cancer diagnosis. In a field experiment with 336 medical students, we manipulated AI deployment in training, in practice, and in both. Our findings reveal that while AI-integrated training and AI assistance independently improved diagnostic performance, their combination yielded the highest accuracy. These results underscore AI's dual role in enhancing human performance through both learning and real-time support, offering insights into AI deployment in professional settings where human expertise remains essential.

Related papers

• Can Domain Experts Rely on AI Appropriately? A Case Study on AI-Assisted Prostate Cancer MRI Diagnosis [19.73932120146401]
  We conduct an in-depth collaboration with radiologists in prostate cancer diagnosis based on MRI images.<n>We develop an interface and conduct two experiments to study how AI assistance and performance feedback shape the decision making of domain experts.
  arXiv  Detail & Related papers  (2025-02-03T18:59:38Z)
• Human-AI Collaborative Game Testing with Vision Language Models [0.0]
  This study investigates how AI can improve game testing by developing and experimenting with an AI-assisted workflow.<n>We evaluate the effectiveness of AI assistance under four conditions: with or without AI support, and with or without detailed knowledge of defects and design documentation.<n>Results indicate that AI assistance significantly improves defect identification performance, particularly when paired with detailed knowledge.
  arXiv  Detail & Related papers  (2025-01-20T23:14:23Z)
• AI-Enhanced Sensemaking: Exploring the Design of a Generative AI-Based Assistant to Support Genetic Professionals [38.54324092761751]
  Generative AI has the potential to transform knowledge work, but further research is needed to understand how knowledge workers envision using and interacting with generative AI.<n>Our research focused on designing a generative AI assistant to aid genetic professionals in analyzing whole genome sequences (WGS) and other clinical data for rare disease diagnosis.
  arXiv  Detail & Related papers  (2024-12-19T22:54:49Z)
• How Performance Pressure Influences AI-Assisted Decision Making [57.53469908423318]
  We show how pressure and explainable AI (XAI) techniques interact with AI advice-taking behavior.<n>Our results show complex interaction effects, with different combinations of pressure and XAI techniques either improving or worsening AI advice taking behavior.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• BO-Muse: A human expert and AI teaming framework for accelerated experimental design [58.61002520273518]
  Our algorithm lets the human expert take the lead in the experimental process. We show that our algorithm converges sub-linearly, at a rate faster than the AI or human alone.
  arXiv  Detail & Related papers  (2023-03-03T02:56:05Z)
• Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making [10.890854857970488]
  Many factors can impact success of Human-AI teams, including a user's domain expertise, mental models of an AI system, trust in recommendations, and more. Our study examined user performance in a non-trivial blood vessel labeling task where participants indicated whether a given blood vessel was flowing or stalled. Our results show that while recommendations from an AI-Assistant can aid user decision making, factors such as users' baseline performance relative to the AI and complementary tuning of AI error types significantly impact overall team performance.
  arXiv  Detail & Related papers  (2022-08-16T21:39:58Z)
• Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging [24.911186503082465]
  This study explores the effects of providing AI assistance at the start of a diagnostic session in radiology versus after the radiologist has made a provisional decision. We found that participants who are asked to register provisional responses in advance of reviewing AI inferences are less likely to agree with the AI regardless of whether the advice is accurate and, in instances of disagreement with the AI, are less likely to seek the second opinion of a colleague.
  arXiv  Detail & Related papers  (2022-05-19T16:59:25Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Learning to Complement Humans [67.38348247794949]
  A rising vision for AI in the open world centers on the development of systems that can complement humans for perceptual, diagnostic, and reasoning tasks. We demonstrate how an end-to-end learning strategy can be harnessed to optimize the combined performance of human-machine teams.
  arXiv  Detail & Related papers  (2020-05-01T20:00:23Z)
• Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork [54.309495231017344]
  We argue that AI systems should be trained in a human-centered manner, directly optimized for team performance. We study this proposal for a specific type of human-AI teaming, where the human overseer chooses to either accept the AI recommendation or solve the task themselves. Our experiments with linear and non-linear models on real-world, high-stakes datasets show that the most accuracy AI may not lead to highest team performance.
  arXiv  Detail & Related papers  (2020-04-27T19:06:28Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.