MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetings

• URL: http://arxiv.org/abs/2607.19235v1
• Date: Tue, 21 Jul 2026 16:05:49 GMT
• Title: MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetings
• Abstract summary: We introduce MeetingToM, a benchmark for complex social behavior reasoning in naturalistic multi-party meetings.<n>The benchmark is hierarchically organized to evaluate ToM at increasing levels of social granularity.<n>Our results highlight key challenges and establish MeetingToM as a testbed for advancing meeting-grounded ToM in multimodal models.
• Score: 11.551333946022053
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Theory of Mind (ToM), the ability to infer other's beliefs, intentions, and states of knowledge, is central to social interaction, yet remains challenging for current Multimodal Large Language Models (MLLMs), especially in multi-party meetings where cues are distributed across speech and behavior. Existing multimodal ToM benchmarks mainly focus on video-grounded question answering over overt, externally verifiable signals, and provide limited coverage of latent social states and group dynamics. We introduce MeetingToM, a benchmark for complex social behavior reasoning in naturalistic multi-party meetings. MeetingToM targets meeting-specific phenomena such as \textbf{pseudo-consensus}, where apparent agreement masks private dissent under social pressure. The benchmark is hierarchically organized to evaluate ToM at increasing levels of social granularity, including (i) subject-level mental state prediction, (ii) dyadic-level addressee understanding, and (iii) group-level consensus reasoning. We provide a unified evaluation protocol and conduct systematic analyses of representative MLLMs, revealing persistent limitations in integrating non-verbal cues, inferring hidden attitudes, and distinguishing genuine consensus from pseudo-consensus. Our results highlight key challenges and establish MeetingToM as a testbed for advancing meeting-grounded ToM in multimodal models.

Related papers

• ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation [65.39532118490511]
  ProMediConv is a novel benchmarking framework that models mediation as a proactive, multi-stage, and party-aware dialogue process.<n>Using 972 complete real-world cases, we construct a high-fidelity mediation dataset with utterance-level annotations of strategies and BP states.<n>We also propose MAD (Mean Attribute Difference), a fine-grained metric that captures BP shifts throughout the dialogue.
  arXiv  Detail & Related papers  (2026-09-10T05:26:40Z)
• Belief Without Behavior: Measuring the Translation of Theory of Mind into Coordinated Social Action in Vision-Language Models [0.509780930114934]
  Effective social interaction requires agents to translate mental state inferences into coordinated behavioral signals across verbal and nonverbal channels simultaneously.<n>We introduce MOSAIC, a benchmark in which two embodied agents interact across cooperative and competitive scenarios.<n>We find that VLMs fail to produce behaviors consistent with the expected outcomes under ToM-order constraints, and that imposing explicit ToM-order constraints produces no reliable behavioral change aligned with the specified reasoning level.
  arXiv  Detail & Related papers  (2026-08-21T10:55:14Z)
• EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness [17.292747703126796]
  We introduce EmoMM, a comprehensive benchmark featuring modality-aligned, conflict, and missing subsets.<n>We propose Conflict-aware Head-level Attention Steering (CHASE), a lightweight mechanism that detects modality conflicts and performs inference-time attention steering.
  arXiv  Detail & Related papers  (2026-05-01T18:35:49Z)
• SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models [86.19617358080016]
  Social Omni is a benchmark that operationalizes the evaluation of conversational interactivity across three core dimensions.<n>Social Omni features 2,000 perception samples and a quality-controlled diagnostic set of 209 interaction-generation instances.<n>Our analysis reveals a pronounced decoupling between a model's perceptual accuracy and its ability to generate contextually appropriate interruptions.
  arXiv  Detail & Related papers  (2026-03-17T17:58:44Z)
• CoMMET: To What Extent Can LLMs Perform Theory of Mind Tasks? [26.35781229730513]
  Theory of Mind (ToM)-the ability to reason about the mental states of oneself and others-is a cornerstone of human social intelligence.<n>Existing benchmarks for assessing ToM in Large Language Models (LLMs) are limited.
  arXiv  Detail & Related papers  (2026-03-12T13:29:42Z)
• Doing Things with Words: Rethinking Theory of Mind Simulation in Large Language Models [48.815314312823006]
  This study explores whether the Generative Agent-Based Model (GABM) Concordia can effectively model Theory of Mind (ToM) within simulated real-world environments.<n>We assess whether this framework successfully simulates ToM abilities and whether GPT-4 can perform tasks by making genuine inferences from social context.
  arXiv  Detail & Related papers  (2025-10-15T10:48:31Z)
• The Hunger Game Debate: On the Emergence of Over-Competition in Multi-Agent Systems [90.96738882568224]
  This paper investigates the over-competition in multi-agent debate, where agents under extreme pressure exhibit unreliable, harmful behaviors.<n>To study this phenomenon, we propose HATE, a novel experimental framework that simulates debates under a zero-sum competition arena.
  arXiv  Detail & Related papers  (2025-09-30T11:44:47Z)
• ToM-SSI: Evaluating Theory of Mind in Situated Social Interactions [17.2191603793837]
  ToM-SSI is a new benchmark specifically designed to test ToM capabilities in environments rich with social interactions and spatial dynamics.<n>This unique design allows us to study, for the first time, mixed cooperative-obstructive settings and reasoning about multiple agents' mental state in parallel.
  arXiv  Detail & Related papers  (2025-09-05T12:58:15Z)
• Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Planner [32.33827730707331]
  We propose a scalable Bayesian ToM planner that decomposes ToM reasoning into stepwise Bayesian updates.<n>Our framework introduces weak-to-strong control, allowing smaller language models to specialize in ToM-specific likelihood estimation.<n>Our method achieves a 4.6% accuracy improvement over state-of-the-art techniques on multimodal ToM benchmarks.
  arXiv  Detail & Related papers  (2025-06-02T04:23:45Z)
• Social Debiasing for Fair Multi-modal LLMs [59.61512883471714]
  Multi-modal Large Language Models (MLLMs) have dramatically advanced the research field and delivered powerful vision-language understanding capabilities.<n>These models often inherit deep-rooted social biases from their training data, leading to uncomfortable responses with respect to attributes such as race and gender.<n>This paper addresses the issue of social biases in MLLMs by introducing a comprehensive counterfactual dataset with multiple social concepts.
  arXiv  Detail & Related papers  (2024-08-13T02:08:32Z)
• NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding [55.38254464415964]
  Theory of mind evaluations currently focuses on testing models using machine-generated data or game settings prone to shortcuts and spurious correlations. We introduce NegotiationToM, a new benchmark designed to stress-test machine ToM in real-world negotiation surrounding covered multi-dimensional mental states.
  arXiv  Detail & Related papers  (2024-04-21T11:51:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.