Understanding LLM Intervention Explanations in Multi-Party Human-Robot Interaction

• URL: http://arxiv.org/abs/2606.29460v1
• Date: Sun, 28 Jun 2026 15:30:15 GMT
• Title: Understanding LLM Intervention Explanations in Multi-Party Human-Robot Interaction
• Abstract summary: Large Language Models (LLMs) are increasingly embedded in social robots to support natural group interactions.<n>This paper investigates the intervention explanations generated by an LLM-based orchestrator in a multi-party interaction.
• Score: 1.916345962680756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) are increasingly embedded in social robots to support natural group interactions, yet their role in complex multi-party settings remains underexplored. In particular, it is unclear how LLM-driven robots decide when and why to intervene in group conversations. This paper investigates the intervention explanations generated by an LLM-based orchestrator in a multi-party interaction involving three human participants and two robots. We conducted a between-subjects study with 24 groups (66 university students), comparing a homogeneous condition (two robots with the same role, i.e., a mover) and a heterogeneous condition (two robots with different roles, i.e., a mover and an opposer). At each conversational turn, the LLM orchestrator decided whether to intervene and generated a textual explanation of its decision. We performed a thematic analysis of 610 intervention explanations, identifying five recurring themes. Results show that explanations are facilitation-oriented, emphasizing agreement, participation, and interaction flow. While patterns remain stable across conditions, role differentiation emerges: the mover supports coordination, whereas the opposer drives goal-oriented interventions. These findings contribute to explainable AI by characterizing how LLM-driven systems justify intervention decisions in real-time, multi-party human-robot interaction.

Related papers

• HiBRIDGE: A Hierarchical Bayesian Neural Network Framework for Interpretable Dialogue Management in Group-Robot Interaction [7.750008457657593]
  HiBRIDGE is a hierarchical Bayesian network framework for group-robot dialogue management.<n>It combines strong predictive performance with a structured decision process that supports more interpretable explanations of robot behaviour.<n>Overall, HiBRIDGE combines strong predictive performance with a structured decision process that supports more interpretable explanations of robot behaviour.
  arXiv  Detail & Related papers  (2026-09-08T12:44:38Z)
• Social Structure Matters in 3D Human-Human Interaction Generation [70.08319433924056]
  We formulate HHI generation as a social structure modeling and grounding problem.<n>The model must first infer how an interaction unfolds and how the two actors coordinate their roles.<n>Our framework produces 3D HHI with improved phase consistency, role alignment, and partner-aware coordination.
  arXiv  Detail & Related papers  (2026-06-23T07:43:13Z)
• CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy [67.76658561040625]
  CHORUS is a framework that adapts a single vision-language-action (VLA) backbone to control diverse, multi-robot teams.<n>In experiments, CHORUS achieves a 64% point improvement over decentralized, from-scratch models.<n>Results show that a shared VLA backbone is capable of achieving decentralized multi-robot collaboration.
  arXiv  Detail & Related papers  (2026-06-10T17:26:08Z)
• A Multimodal Framework for Human-Multi-Agent Interaction [5.410329948686681]
  We introduce a multimodal framework for human-multi-agent interaction in which each robot operates as an autonomous cognitive agent.<n>At the team level, a centralized coordination mechanism regulates turn-taking and agent participation to prevent overlapping speech.<n>Future work will focus on larger-scale user studies and deeper exploration of socially grounded multi-agent interaction dynamics.
  arXiv  Detail & Related papers  (2026-03-24T14:35:40Z)
• Cooperation Through Indirect Reciprocity in Child-Robot Interactions [81.62347137438248]
  We investigate whether indirect reciprocity can be transposed to children-robot interactions.<n>We find that IR extends to children and robots solving coordination dilemmas.<n>We observe that cooperating through multi-armed bandit algorithms is highly dependent on the strategies revealed by humans.
  arXiv  Detail & Related papers  (2025-11-07T07:08:32Z)
• RoboOmni: Proactive Robot Manipulation in Omni-modal Context [165.09049429566238]
  We introduce cross-modal contextual instructions, a new setting where intent is derived from spoken dialogue, environmental sounds, and visual cues rather than explicit commands.<n>We present RoboOmni, a framework based on end-to-end omni-modal LLMs that unifies intention recognition, interaction confirmation, and action execution.<n>Experiments in simulation and real-world settings show that Robo Omni surpasses text- and ASR-based baselines in success rate, inference speed, intention recognition, and proactive assistance.
  arXiv  Detail & Related papers  (2025-10-27T18:49:03Z)
• Let's Roleplay: Examining LLM Alignment in Collaborative Dialogues [11.888786446094057]
  This paper examines how different alignment methods affect Large Language Models' effectiveness as partners in multiturn, multiparty collaborations.<n>Using a roleplay methodology, we evaluate interventions from differently-trained friction agents in collaborative task conversations.<n>Our results show that a friction-aware approach significantly outperforms common alignment baselines in helping both convergence to a common ground, or agreed-upon task-relevant propositions, and correctness of task outcomes.
  arXiv  Detail & Related papers  (2025-09-07T00:58:10Z)
• Whom to Respond To? A Transformer-Based Model for Multi-Party Social Robot Interaction [4.276453870301421]
  We propose a Transformer-based multi-task learning framework to improve the decision-making process of social robots.<n>We construct a novel multi-party HRI dataset that captures real-world complexities, such as gaze misalignment.<n>Our findings contribute to the development of socially intelligent social robots capable of engaging in natural and context-aware multi-party interactions.
  arXiv  Detail & Related papers  (2025-07-15T03:42:14Z)
• Moderating Group Conversation Dynamics with Social Robots [0.0]
  The study involved 300 participants, divided into groups of four, interacting with a humanoid robot serving as the moderator. The robot utilized conversation data to determine the most appropriate speaker to address.
  arXiv  Detail & Related papers  (2024-07-31T20:29:20Z)
• SocialBench: Sociality Evaluation of Role-Playing Conversational Agents [85.6641890712617]
  Large language models (LLMs) have advanced the development of various AI conversational agents. SocialBench is the first benchmark designed to evaluate the sociality of role-playing conversational agents at both individual and group levels. We find that agents excelling in individual level does not imply their proficiency in group level.
  arXiv  Detail & Related papers  (2024-03-20T15:38:36Z)
• Interpreting Neural Policies with Disentangled Tree Representations [58.769048492254555]
  We study interpretability of compact neural policies through the lens of disentangled representation. We leverage decision trees to obtain factors of variation for disentanglement in robot learning. We introduce interpretability metrics that measure disentanglement of learned neural dynamics.
  arXiv  Detail & Related papers  (2022-10-13T01:10:41Z)
• Controlling the Sense of Agency in Dyadic Robot Interaction: An Active Inference Approach [6.421670116083633]
  We examine dyadic imitative interactions of robots using a variational recurrent neural network model. We examined how regulating the complexity term to minimize free energy during training determines the dynamic characteristics of networks.
  arXiv  Detail & Related papers  (2021-03-03T02:38:09Z)
• Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Driving [47.22329993674051]
  We aim to develop an explainable model that generates explanations consistent with both human domain knowledge and the model's inherent causal relation. In particular, we focus on an essential building block of autonomous driving, multi-agent interaction modeling. We demonstrate that it can model interactive traffic scenarios under both simulation and real-world settings.
  arXiv  Detail & Related papers  (2021-02-23T19:34:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.