Are you with me? A Framework for Detecting Mental Model Discrepancies in Task-Based Team Dialogues

• URL: http://arxiv.org/abs/2605.03149v1
• Date: Mon, 04 May 2026 20:42:16 GMT
• Title: Are you with me? A Framework for Detecting Mental Model Discrepancies in Task-Based Team Dialogues
• Abstract summary: We propose a framework to identify and categorize four types of mental model discrepancies: unsupported beliefs, false beliefs, belief contradictions, and omissions.<n>Using dialogues from twenty dyad teams performing collaborative object identification tasks, we demonstrate that these discrepancy patterns contain predictive signals.
• Score: 3.771395176745804
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans typically use natural language to update teammates on task states. Since not all updates are communicated, discrepancies arise between the team members' mental models that negatively affect overall team performance. How can we categorize such discrepancies? Do misalignments detected in team dialogue predict future mental model misalignments? Traditional shared mental model (SMM) assessment methods rely on retrospective expert coding that cannot capture real-time coordination dynamics. We propose a framework to identify and categorize four types of mental model discrepancies: unsupported beliefs, false beliefs, belief contradictions, and omissions, all of which can naturally emerge in team dialogues. Using dialogues from twenty dyad teams performing collaborative object identification tasks across four sequential levels, we demonstrate that these discrepancy patterns contain predictive signals. Averaging historical discrepancy counts achieves meaningful prediction accuracy using uniform weighting as an exploratory baseline, with differential predictability across discrepancy types.

Related papers

• CompanionBench: A Theory-Anchored, Real-World-Grounded Benchmark for AI Emotional Companionship [7.643253506288442]
  Existing benchmarks use hand-authored scenarios and prompted simulators.<n>We introduce CompanionBench, an interactive bilingual benchmark.<n>It is the first companion benchmark to ground both its scenarios and a trained user simulator in real-world data.
  arXiv  Detail & Related papers  (2026-08-03T10:44:04Z)
• LLM Reasoning Predicts When Models Are Right: Evidence from Coding Classroom Discourse [0.18268488712787334]
  Large Language Models (LLMs) are increasingly deployed to automatically label and analyze educational dialogue at scale.<n>We investigate whether reasoning generated by LLMs can be used to predict the correctness of a model's own predictions.<n>We analyze 30,300 teacher utterances from classroom dialogue, each labeled by multiple state-of-the-art LLMs with an instructional move construct and an accompanying reasoning.
  arXiv  Detail & Related papers  (2026-02-10T14:38:13Z)
• Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generation [48.63200319578052]
  We develop a taxonomy of types of ambiguity that arise in this task and propose metrics to quantify them.<n>Our work also explores how multi-turn dialogue can reduce ambiguity, therefore, improve code accuracy by better matching user goals.
  arXiv  Detail & Related papers  (2025-10-10T13:44:40Z)
• WHEN TO ACT, WHEN TO WAIT: Modeling the Intent-Action Alignment Problem in Dialogue [23.39224016484548]
  Dialogue systems often fail when user utterances are semantically complete yet lack clarity and completeness required for appropriate system action.<n>We present STORM, a framework modeling asymmetric information dynamics through conversations between UserLLM and AgentLLM.<n>Our contributions include: (1) formalizing asymmetric information processing in dialogue systems; (2) modeling intent formation tracking collaborative understanding evolution; and (3) evaluation metrics measuring internal cognitive improvements alongside task performance.
  arXiv  Detail & Related papers  (2025-06-02T17:11:10Z)
• Predicting Team Performance from Communications in Simulated Search-and-Rescue [1.2289361708127877]
  We analyze conversational data to identify team traits and their correlation with teaming outcomes.<n>Our findings show that variations in teaming outcomes can be explained through these inferences.
  arXiv  Detail & Related papers  (2025-03-05T07:20:27Z)
• ML-SPEAK: A Theory-Guided Machine Learning Method for Studying and Predicting Conversational Turn-taking Patterns [25.049072387358244]
  We develop a computational model of conversational turn-taking within self-organized teams. By bridging the gap between individual personality traits and team communication patterns, our model has the potential to inform theories of team processes.
  arXiv  Detail & Related papers  (2024-11-23T01:27:01Z)
• MindDial: Belief Dynamics Tracking with Theory-of-Mind Modeling for Situated Neural Dialogue Generation [62.44907105496227]
  MindDial is a novel conversational framework that can generate situated free-form responses with theory-of-mind modeling. We introduce an explicit mind module that can track the speaker's belief and the speaker's prediction of the listener's belief. Our framework is applied to both prompting and fine-tuning-based models, and is evaluated across scenarios involving both common ground alignment and negotiation.
  arXiv  Detail & Related papers  (2023-06-27T07:24:32Z)
• Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue [92.01165203498299]
  Embodied dialogue instruction following requires an agent to complete a complex sequence of tasks from a natural language exchange. This paper argues that imitation learning (IL) and related low-level metrics are actually misleading and do not align with the goals of embodied dialogue research.
  arXiv  Detail & Related papers  (2022-10-10T05:51:40Z)
• I Beg to Differ: A study of constructive disagreement in online conversations [15.581515781839656]
  We construct a corpus of 7 425 Wikipedia Talk page conversations that contain content disputes. We define the task of predicting whether disagreements will be escalated to mediation by a moderator. We develop a variety of neural models and show that taking into account the structure of the conversation improves predictive accuracy.
  arXiv  Detail & Related papers  (2021-01-26T16:36:43Z)
• I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling [104.09033240889106]
  We introduce the DialoguE COntradiction DEtection task (DECODE) and a new conversational dataset containing both human-human and human-bot contradictory dialogues. We then compare a structured utterance-based approach of using pre-trained Transformer models for contradiction detection with the typical unstructured approach.
  arXiv  Detail & Related papers  (2020-12-24T18:47:49Z)
• Probing Task-Oriented Dialogue Representation from Language Models [106.02947285212132]
  This paper investigates pre-trained language models to find out which model intrinsically carries the most informative representation for task-oriented dialogue tasks. We fine-tune a feed-forward layer as the classifier probe on top of a fixed pre-trained language model with annotated labels in a supervised way.
  arXiv  Detail & Related papers  (2020-10-26T21:34:39Z)
• Learning an Unreferenced Metric for Online Dialogue Evaluation [53.38078951628143]
  We propose an unreferenced automated evaluation metric that uses large pre-trained language models to extract latent representations of utterances. We show that our model achieves higher correlation with human annotations in an online setting, while not requiring true responses for comparison during inference.
  arXiv  Detail & Related papers  (2020-05-01T20:01:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.