Beyond "ChatGPT Can Make Mistakes": Designing Interventions to Support Metacognitive Monitoring in AI-Assisted Work

• URL: http://arxiv.org/abs/2609.17065v1
• Date: Tue, 15 Sep 2026 12:09:18 GMT
• Title: Beyond "ChatGPT Can Make Mistakes": Designing Interventions to Support Metacognitive Monitoring in AI-Assisted Work
• Abstract summary: AI assistance places a metacognitive demand on users, who must judge their own competence and the system's.<n>We elicited 30 interventions from 11 experts and organized them into a design space of time.<n>We contribute a shared vocabulary, a design space, and evidence that measured monitoring and task performance are separable design targets.
• Score: 21.306270561292454
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: AI assistance places a metacognitive demand on users, who must judge their own competence and the system's. Yet designers lack comparative evidence on which interventions to choose, where to place them, and how to tell whether they worked. We elicited 30 interventions from 11 experts and, with prior work, organized them into a design space of time (when an intervention acts), level (whose competence is judged), and source (who supplies the monitoring cue). A between-subjects experiment (N = 917; 12 planning-and-organizing problems) compared a per-task reliability card, contrasting replies, pause points, and post-problem reflection against a baseline LLM assistant. Reliability cards and contrasting replies reduced estimation error and overconfidence and increased aggregate confidence discrimination. No task-performance improvement or average within-item discrimination gain was established. We contribute a shared vocabulary, a design space, and evidence that measured monitoring and task performance are separable design targets.

Related papers

• Factors Impacting Developer Efficiency: Results from an Adaptive Longitudinal Study [0.8373057326694192]
  This study investigates the primary factors hindering the perceived efficiency of developers in a consulting and professional development context.<n>We conducted a mixed-methods longitudinal case study applying the Adaptive Developer Efficiency Monitoring Method (ADEMM) to 27 external software developers.
  arXiv  Detail & Related papers  (2026-08-17T13:53:03Z)
• ADEMM: A Longitudinal Method for Monitoring Developer Efficiency in Industry [0.8373057326694192]
  This study proposes and evaluates the Adaptive Developer Efficiency Monitoring Method (ADEMM)<n>ADEMM is an adaptive longitudinal method for monitoring developer efficiency when the monitoring organization does not directly employ the developers.
  arXiv  Detail & Related papers  (2026-08-17T13:42:30Z)
• Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security [57.35851886874902]
  Agentic AI systems execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that challenge trustworthiness.<n>This survey provides a focused examination of trustworthy agentic AI through two core dimensions that are critical for high-risk deployments.<n>For each dimension, we clarify key concepts, identify where risks emerge along the agent workflow, and summarize stage-targeted mitigation strategies.
  arXiv  Detail & Related papers  (2026-05-17T10:26:37Z)
• Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems [65.90791804095561]
  Multi-agent systems address this through structured collaboration among specialized agents, but tighter coordination amplifies a less explored risk.<n>Existing surveys cover individual agent capabilities, multi-agent collaboration, or agent self-evolution separately, leaving the causal dependencies among them unexamined.<n>This survey provides a unified review organized around four causally linked stages, which we term the LIFE progression: Lay the capability foundation, Integrate agents through collaboration, Find faults through attribution, and Evolve through autonomous self-improvement.
  arXiv  Detail & Related papers  (2026-05-14T14:36:13Z)
• Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs [51.56484100374058]
  guardrail design entails explicit trade-offs among adversarial robustness, benign-task usability, and response latency.<n>We evaluate a domain-specific multi-layer safeguard pipeline combining deterministic pattern filters, structural validation, contextual sandboxing, and session-level behavioral checks.<n>Results expose operational trade-offs: NeMo reaches 0 percent bypass at 16.22 percent FPR and roughly 1.5s latency, while Prompt Guard yields 38.48 percent bypass with 3.60 percent FPR.
  arXiv  Detail & Related papers  (2026-03-29T18:52:01Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• Bridging Expertise Gaps: The Role of LLMs in Human-AI Collaboration for Cybersecurity [17.780795900414716]
  This study investigates whether large language models (LLMs) can function as intelligent collaborators to bridge expertise gaps in cybersecurity decision-making.<n>We find that human-AI collaboration improves task performance,reducing false positives in phishing detection and false negatives in intrusion detection.
  arXiv  Detail & Related papers  (2025-05-06T04:47:52Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• AI for human assessment: What do professional assessors need? [33.88509725285237]
  This case study aims to help professional assessors make decisions in human assessment, in which they conduct interviews with assessees and evaluate their suitability for certain job roles. A computational system that can extract nonverbal cues of assesses would be beneficial to assessors in terms of supporting their decision making. We developed such a system based on an unsupervised anomaly detection algorithm using multimodal behavioral features such as facial keypoints, pose, head pose, and gaze.
  arXiv  Detail & Related papers  (2022-04-18T03:35:37Z)
• Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support [18.148737010217953]
  We conduct interviews and workshops with AI practitioners to identify practitioners' processes, challenges, and needs for support. We find that practitioners face challenges when choosing performance metrics, identifying the most relevant direct stakeholders and demographic groups. We identify impacts on fairness work stemming from a lack of engagement with direct stakeholders, business imperatives that prioritize customers over marginalized groups, and the drive to deploy AI systems at scale.
  arXiv  Detail & Related papers  (2021-12-10T17:14:34Z)
• Robust Learning Through Cross-Task Consistency [92.42534246652062]
  We propose a broadly applicable and fully computational method for augmenting learning with Cross-Task Consistency. We observe that learning with cross-task consistency leads to more accurate predictions and better generalization to out-of-distribution inputs.
  arXiv  Detail & Related papers  (2020-06-07T09:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.