Model Reconciliation through Explainability and Collaborative Recovery in Assistive Robotics

• URL: http://arxiv.org/abs/2601.06552v3
• Date: Wed, 04 Feb 2026 15:42:19 GMT
• Title: Model Reconciliation through Explainability and Collaborative Recovery in Assistive Robotics
• Abstract summary: We leverage a Large Language Model to predict and explain the difference between the robot's and the human's mental models.<n>Our framework aims to solve the model divergence after the explanation by allowing the human to correct the robot.
• Score: 4.666406865641358
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Whenever humans and robots work together, it is essential that unexpected robot behavior can be explained to the user. Especially in applications such as shared control the user and the robot must share the same model of the objects in the world, and the actions that can be performed on these objects. In this paper, we achieve this with a so-called model reconciliation framework. We leverage a Large Language Model to predict and explain the difference between the robot's and the human's mental models, without the need of a formal mental model of the user. Furthermore, our framework aims to solve the model divergence after the explanation by allowing the human to correct the robot. We provide an implementation in an assistive robotics domain, where we conduct a set of experiments with a real wheelchair-based mobile manipulator and its digital twin.

Related papers

• AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization [88.25615187835321]
  We propose AnyWorld, a cross-embodiment world modeling framework.<n>Our model factorizes an interaction into action, camera, and embodiment.<n>We train the model with large-scale human interaction pretraining.
  arXiv  Detail & Related papers  (2026-08-29T12:55:15Z)
• Autonomous Human-Robot Interaction via Operator Imitation [3.650193138379926]
  We propose to create autonomous interactive robots, by training a model to imitate operator data.<n>Our model is trained on a dataset of human-robot interactions.<n>We show that our method enables simple autonomous human-robot interactions comparable to the expert-operator baseline.
  arXiv  Detail & Related papers  (2025-04-03T16:06:44Z)
• Controlling diverse robots by inferring Jacobian fields with deep networks [48.279199537720714]
  Mirroring the complex structures and diverse functions of natural organisms is a long-standing challenge in robotics.<n>We introduce a method that uses deep neural networks to map a video stream of a robot to its visuomotor Jacobian field.<n>Our approach achieves accurate closed-loop control and recovers the causal dynamic structure of each robot.
  arXiv  Detail & Related papers  (2024-07-11T17:55:49Z)
• Singing the Body Electric: The Impact of Robot Embodiment on User Expectations [7.408858358967414]
  Users develop mental models of robots to conceptualize what kind of interactions they can have with those robots. conceptualizations are often formed before interactions with the robot and are based only on observing the robot's physical design. We propose to use multimodal features of robot embodiments to predict what kinds of expectations users will have about a given robot's social and physical capabilities.
  arXiv  Detail & Related papers  (2024-01-13T04:42:48Z)
• InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions [7.574421886354134]
  InteRACT architecture pre-trains a conditional intent prediction model on large human-human datasets and fine-tunes on a small human-robot dataset. We evaluate on a set of real-world collaborative human-robot manipulation tasks and show that our conditional model improves over various marginal baselines.
  arXiv  Detail & Related papers  (2023-11-21T19:15:17Z)
• ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space [9.806227900768926]
  This paper introduces a novel deep-learning approach for human-to-robot motion. Our method does not require paired human-to-robot data, which facilitates its translation to new robots. Our model outperforms existing works regarding human-to-robot similarity in terms of efficiency and precision.
  arXiv  Detail & Related papers  (2023-09-11T08:55:04Z)
• A Capability and Skill Model for Heterogeneous Autonomous Robots [69.50862982117127]
  capability modeling is considered a promising approach to semantically model functions provided by different machines. This contribution investigates how to apply and extend capability models from manufacturing to the field of autonomous robots.
  arXiv  Detail & Related papers  (2022-09-22T10:13:55Z)
• Robots with Different Embodiments Can Express and Influence Carefulness in Object Manipulation [104.5440430194206]
  This work investigates the perception of object manipulations performed with a communicative intent by two robots. We designed the robots' movements to communicate carefulness or not during the transportation of objects.
  arXiv  Detail & Related papers  (2022-08-03T13:26:52Z)
• Model Elicitation through Direct Questioning [22.907680615911755]
  We show how a robot can interact to localize the human model from a set of models. We show how to generate questions to refine the robot's understanding of the teammate's model.
  arXiv  Detail & Related papers  (2020-11-24T18:17:16Z)
• Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks [83.37025218216888]
  We propose a novel explainable AI (XAI) framework for achieving human-like communication in human-robot collaborations. The robot builds a hierarchical mind model of the human user and generates explanations of its own mind as a form of communications. Results show that the generated explanations of our approach significantly improves the collaboration performance and user perception of the robot.
  arXiv  Detail & Related papers  (2020-07-24T23:35:03Z)
• Learning Predictive Models From Observation and Interaction [137.77887825854768]
  Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works. However, learning a model that captures the dynamics of complex skills represents a major challenge. We propose a method to augment the training set with observational data of other agents, such as humans.
  arXiv  Detail & Related papers  (2019-12-30T01:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.