A Taxonomy of Questions for Critical Reflection in Machine-Assisted Decision-Making

• URL: http://arxiv.org/abs/2504.12830v3
• Date: Tue, 26 Aug 2025 13:23:02 GMT
• Title: A Taxonomy of Questions for Critical Reflection in Machine-Assisted Decision-Making
• Abstract summary: Decision-makers run the risk of relying too much on machine recommendations.<n> Reflection has been shown to increase cognitive engagement and improve critical thinking.<n>We present a taxonomy of questions aimed at promoting reflection and cognitive engagement.
• Score: 3.884856136722027
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Decision-makers run the risk of relying too much on machine recommendations, which is associated with lower cognitive engagement. Reflection has been shown to increase cognitive engagement and improve critical thinking and therefore decision-making. Questions are a means to stimulate reflection, but there is a research gap regarding the systematic creation and use of relevant questions for machine-assisted decision-making. We therefore present a taxonomy of questions aimed at promoting reflection and cognitive engagement in order to stimulate a deliberate decision-making process. Our taxonomy builds on the Socratic questioning method and a question bank for explainable AI. As a starting point, we focus on clinical decision-making. Brief discussions with two medical and three educational researchers provide feedback on the relevance and expected benefits of our taxonomy. Our work contributes to research on mitigating overreliance in human-AI interactions and aims to support effective human oversight as required by the European AI Act.

Related papers

• Disrupting Cognitive Passivity: Rethinking AI-Assisted Data Literacy through Cognitive Alignment [17.265229093607676]
  We argue that disrupting cognitive passivity requires more dynamic and adaptive strategy through cognitive alignment.<n>In the framework, we provide the mapping between AI's interaction mode (transmissive or deliberative) and users' cognitive demand (receptive or deliberative)
  arXiv  Detail & Related papers  (2026-04-03T06:47:55Z)
• Promoting Critical Thinking With Domain-Specific Generative AI Provocations [11.57437441019857]
  Some argue that AI-driven provocations, such as questions asking for human clarification and justification, are beneficial for eliciting critical thinking.<n>We reflect on how design decisions shape the form and effectiveness of such provocations.<n>Our observations and user feedback suggest that domain-specific provocations, implemented through productive friction and interactions that depend on user contribution, can meaningfully support critical thinking.
  arXiv  Detail & Related papers  (2026-03-20T14:20:05Z)
• Intelligent Reasoning Cues: A Framework and Case Study of the Roles of AI Information in Complex Decisions [10.853817540556348]
  We study the role of eight types of reasoning cues in a high-stakes clinical decision.<n>We find that reasoning cues have distinct patterns of influence that can directly inform design.<n>Our results suggest that reasoning cues should prioritize tasks with high variability and discretion, adapt to ensure compatibility with evolving decision needs, and provide complementary, rigorous insights on complex cases.
  arXiv  Detail & Related papers  (2026-01-30T19:22:23Z)
• Towards a Cascaded LLM Framework for Cost-effective Human-AI Decision-Making [55.2480439325792]
  We present a cascaded LLM decision framework that adaptively delegates tasks across multiple tiers of expertise.<n>First, a deferral policy determines whether to accept the base model's answer or regenerate it with the large model.<n>Second, an abstention policy decides whether the cascade model response is sufficiently certain or requires human intervention.
  arXiv  Detail & Related papers  (2025-06-13T15:36:22Z)
• Must Read: A Systematic Survey of Computational Persuasion [60.83151988635103]
  AI-driven persuasion can be leveraged for beneficial applications, but also poses threats through manipulation and unethical influence.<n>Our survey outlines future research directions to enhance the safety, fairness, and effectiveness of AI-powered persuasion.
  arXiv  Detail & Related papers  (2025-05-12T17:26:31Z)
• Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisions [31.11473190744529]
  A growing body of work in Ethical AI attempts to capture human moral judgments through simple computational models.<n>We conducted twenty interviews where participants explained their rationale for their judgments about who should receive a kidney.<n>We observe participants: (a) value patients' morally-relevant attributes to different degrees; (b) use diverse decision-making processes, citings to reduce decision complexity; (c) can change their opinions; and (e) express enthusiasm and concern regarding AI assisting humans in kidney allocation decisions.
  arXiv  Detail & Related papers  (2025-03-02T15:42:17Z)
• Engaging with AI: How Interface Design Shapes Human-AI Collaboration in High-Stakes Decision-Making [8.948482790298645]
  We examine how various decision-support mechanisms impact user engagement, trust, and human-AI collaborative task performance.<n>Our findings reveal that mechanisms like AI confidence levels, text explanations, and performance visualizations enhanced human-AI collaborative task performance.
  arXiv  Detail & Related papers  (2025-01-28T02:03:00Z)
• How Performance Pressure Influences AI-Assisted Decision Making [57.53469908423318]
  We show how pressure and explainable AI (XAI) techniques interact with AI advice-taking behavior.<n>Our results show complex interaction effects, with different combinations of pressure and XAI techniques either improving or worsening AI advice taking behavior.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Combining AI Control Systems and Human Decision Support via Robustness and Criticality [53.10194953873209]
  We extend a methodology for adversarial explanations (AE) to state-of-the-art reinforcement learning frameworks. We show that the learned AI control system demonstrates robustness against adversarial tampering. In a training / learning framework, this technology can improve both the AI's decisions and explanations through human interaction.
  arXiv  Detail & Related papers  (2024-07-03T15:38:57Z)
• The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoning [70.16523526957162]
  Understanding commonsense causality helps people understand the principles of the real world better. Despite its significance, a systematic exploration of this topic is notably lacking. Our work aims to provide a systematic overview, update scholars on recent advancements, and provide a pragmatic guide for beginners.
  arXiv  Detail & Related papers  (2024-06-27T16:30:50Z)
• Distilling Reasoning Ability from Large Language Models with Adaptive Thinking [54.047761094420174]
  Chain of thought finetuning (cot-finetuning) aims to endow small language models (SLM) with reasoning ability to improve their performance towards specific tasks. Most existing cot-finetuning methods adopt a pre-thinking mechanism, allowing the SLM to generate a rationale before providing an answer. This mechanism enables SLM to analyze and think about complex questions, but it also makes answer correctness highly sensitive to minor errors in rationale. We propose a robust post-thinking mechanism to generate answers before rationale.
  arXiv  Detail & Related papers  (2024-04-14T07:19:27Z)
• Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making [47.33241893184721]
  In AI-assisted decision-making, humans often passively review AI's suggestion and decide whether to accept or reject it as a whole.<n>We propose Human-AI Deliberation, a novel framework to promote human reflection and discussion on conflicting human-AI opinions in decision-making.<n>Based on theories in human deliberation, this framework engages humans and AI in dimension-level opinion elicitation, deliberative discussion, and decision updates.
  arXiv  Detail & Related papers  (2024-03-25T14:34:06Z)
• Beyond Recommender: An Exploratory Study of the Effects of Different AI Roles in AI-Assisted Decision Making [48.179458030691286]
  We examine three AI roles: Recommender, Analyzer, and Devil's Advocate. Our results show each role's distinct strengths and limitations in task performance, reliance appropriateness, and user experience. These insights offer valuable implications for designing AI assistants with adaptive functional roles according to different situations.
  arXiv  Detail & Related papers  (2024-03-04T07:32:28Z)
• Toward enriched Cognitive Learning with XAI [44.99833362998488]
  We introduce an intelligent system (CL-XAI) for Cognitive Learning which is supported by artificial intelligence (AI) tools. The use of CL-XAI is illustrated with a game-inspired virtual use case where learners tackle problems to enhance problem-solving skills.
  arXiv  Detail & Related papers  (2023-12-19T16:13:47Z)
• Learning to Make Adherence-Aware Advice [8.419688203654948]
  This paper presents a sequential decision-making model that takes into account the human's adherence level. We provide learning algorithms that learn the optimal advice policy and make advice only at critical time stamps.
  arXiv  Detail & Related papers  (2023-10-01T23:15:55Z)
• Analyzing Character and Consciousness in AI-Generated Social Content: A Case Study of Chirper, the AI Social Network [0.0]
  The study embarks on a comprehensive exploration of AI behavior, analyzing the effects of diverse settings on Chirper's responses. Through a series of cognitive tests, the study gauges the self-awareness and pattern recognition prowess of Chirpers. An intriguing aspect of the research is the exploration of the potential influence of a Chirper's handle or personality type on its performance.
  arXiv  Detail & Related papers  (2023-08-30T15:40:18Z)
• Perspectives on Large Language Models for Relevance Judgment [56.935731584323996]
  Large language models (LLMs) claim that they can assist with relevance judgments. It is not clear whether automated judgments can reliably be used in evaluations of retrieval systems.
  arXiv  Detail & Related papers  (2023-04-13T13:08:38Z)
• Fairness in Agreement With European Values: An Interdisciplinary Perspective on AI Regulation [61.77881142275982]
  This interdisciplinary position paper considers various concerns surrounding fairness and discrimination in AI, and discusses how AI regulations address them. We first look at AI and fairness through the lenses of law, (AI) industry, sociotechnology, and (moral) philosophy, and present various perspectives. We identify and propose the roles AI Regulation should take to make the endeavor of the AI Act a success in terms of AI fairness concerns.
  arXiv  Detail & Related papers  (2022-06-08T12:32:08Z)
• Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies [79.60322329952453]
  We show how to develop interpretable representations of how agents make decisions. By understanding the decision-making processes underlying a set of observed trajectories, we cast the policy inference problem as the inverse to this online learning problem. We introduce a practical algorithm for retrospectively estimating such perceived effects, alongside the process through which agents update them. Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time.
  arXiv  Detail & Related papers  (2022-03-14T17:40:42Z)
• Intelligent Decision Assistance Versus Automated Decision-Making: Enhancing Knowledge Work Through Explainable Artificial Intelligence [0.0]
  We propose a new class of DSS, namely Intelligent Decision Assistance (IDA) IDA supports knowledge workers without influencing them through automated decision-making. Specifically, we propose to use techniques of Explainable AI (XAI) while withholding concrete AI recommendations.
  arXiv  Detail & Related papers  (2021-09-28T15:57:21Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)
• To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making [4.877174544937129]
  People supported by AI-powered decision support tools frequently overrely on the AI. Adding explanations to the AI decisions does not appear to reduce the overreliance. Our research suggests that human cognitive motivation moderates the effectiveness of explainable AI solutions.
  arXiv  Detail & Related papers  (2021-02-19T00:38:53Z)
• Thinking Fast and Slow in AI [38.8581204791644]
  This paper proposes a research direction to advance AI which draws inspiration from cognitive theories of human decision making. The premise is that if we gain insights about the causes of some human capabilities that are still lacking in AI, we may obtain similar capabilities in an AI system.
  arXiv  Detail & Related papers  (2020-10-12T20:10:05Z)
• Implications of Human Irrationality for Reinforcement Learning [26.76732313120685]
  We argue that human decision making may be a better source of ideas for constraining how machine learning problems are defined than would otherwise be the case. One promising idea concerns human decision making that is dependent on apparently irrelevant aspects of the choice context. We propose a novel POMDP model for contextual choice tasks and show that, despite the apparent irrationalities, a reinforcement learner can take advantage of the way that humans make decisions.
  arXiv  Detail & Related papers  (2020-06-07T07:44:53Z)
• A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores [85.12096045419686]
  We study the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We first show that humans do alter their behavior when the tool is deployed. We show that humans are less likely to adhere to the machine's recommendation when the score displayed is an incorrect estimate of risk.
  arXiv  Detail & Related papers  (2020-02-19T07:27:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.