Structured LLM Reasoning for Zero-Shot Human--Robot Coordination Under Hidden Goals

• URL: http://arxiv.org/abs/2608.04309v1
• Date: Wed, 05 Aug 2026 00:40:03 GMT
• Title: Structured LLM Reasoning for Zero-Shot Human--Robot Coordination Under Hidden Goals
• Abstract summary: We present a structured large-language-model (LLM) architecture for zero-shot human--robot coordination in a cooperative construction task with private goal views.<n> Guided by a Dec-POMDP formulation, the architecture decomposes decision-making into (i) action-conditioned Theory-of-Mind (ToM) inference, (ii) hierarchical planning, (iii) conversation interpretation, (iv) action verification, and (v) feedback-based replanning.
• Score: 3.158926173252809
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a structured large-language-model (LLM) architecture for zero-shot human--robot coordination in a cooperative construction task with private goal views. Guided by a Dec-POMDP formulation, the architecture decomposes decision-making into (i) action-conditioned Theory-of-Mind (ToM) inference, (ii) hierarchical planning, (iii) conversation interpretation, (iv) action verification, and (v) feedback-based replanning. We compare the proposed method with an ablation without ToM inference and a multi-agent reinforcement-learning policy trained offline over many goal pairs. In human-participant experiments, the proposed method required fewer interaction steps and yielded higher post-interaction trust ratings than both baselines. These results suggest that systematically decomposing the team decision problem, using LLMs as tractable surrogates for otherwise intractable inference and planning computations, and retaining conventional verification for physical feasibility can improve both task coordination and the human experience.

Related papers

• Assistance Without Interruption: A Benchmark and LLM-based Framework for Non-Intrusive Human-Robot Assistance [44.10105182063131]
  We formalize and benchmark the non-intrusive assistance as an independent paradigm of human-robot interaction (HRI)<n>Unlike conventional HRI tasks that rely on direct commands, explicit negotiation, or proactive interventions based on user habits and history, our task treats the human's plan as the primary process and formulates assistance as a joint decision over when to act and what to do.<n> Comprehensive experiments on both NIABench and real-world scenarios demonstrate that our method achieves proactive, non-intrusive assistance that reduces human effort while preserving task effectiveness.
  arXiv  Detail & Related papers  (2026-05-02T10:25:40Z)
• When Contextual Inference Fails: Cancelability in Interactive Instruction Following [51.2195840589474]
  We introduce Build What I Mean, an interactive benchmark for contextual meaning construction.<n>In BWIM, models must resolve ambiguity by either performing a contextual inference or requesting clarification at a small communication cost.<n>We observe suboptimal strategies, such as partner-blind over-clarification and question-averse guessing under uncertainty.
  arXiv  Detail & Related papers  (2026-03-20T14:46:59Z)
• DeReCo: Decoupling Representation and Coordination Learning for Object-Adaptive Decentralized Multi-Robot Cooperative Transport [4.030781059424759]
  DeReCo is a novel framework that decouples representation and coordination learning for object-adaptive multi-robot cooperative transport.<n> Experimental results show that DeReCo outperforms baselines in simulation on three training objects, generalizes to six unseen objects with varying masses and friction coefficients, and achieves superior performance on two unseen objects in real-robot experiments.
  arXiv  Detail & Related papers  (2026-03-09T08:53:11Z)
• Uncertainty Mitigation and Intent Inference: A Dual-Mode Human-Machine Joint Planning System [16.75873002054227]
  We propose a unified human-robot joint planning system to tackle dual sources of uncertainty.<n>First, an uncertainty-mitigation joint planning module enables two-way conversations to resolve semantic ambiguity and object uncertainty.<n>Second, a real-time intent-aware collaboration module maintains a probabilistic belief over the human's latent task intent.
  arXiv  Detail & Related papers  (2026-03-08T21:43:30Z)
• Replanning Human-Robot Collaborative Tasks with Vision-Language Models via Semantic and Physical Dual-Correction [7.668882674013904]
  Human-Robot Collaboration plays an important role in assembly tasks by enabling robots to plan and adjust their motions based on interactive, real-time human instructions.<n>Such instructions are often linguistically ambiguous and underspecified, making it difficult to generate physically feasible and cooperative robot behaviors.<n>We propose an HRC framework that augments a VLM-based reasoning with a dual-correction mechanism.
  arXiv  Detail & Related papers  (2026-02-16T08:24:19Z)
• Interaction-Grounded Learning for Contextual Markov Decision Processes with Personalized Feedback [59.287761696290865]
  We propose a computationally efficient algorithm that achieves a sublinear regret guarantee for contextual episodic Markov Decision Processes (MDPs) with personalized feedback.<n>We demonstrate the effectiveness of our method in learning personalized objectives from multi-turn interactions through experiments on both a synthetic episodic MDP and a real-world user booking dataset.
  arXiv  Detail & Related papers  (2026-02-09T06:29:54Z)
• Agentic Reasoning for Large Language Models [122.81018455095999]
  Reasoning is a fundamental cognitive process underlying inference, problem-solving, and decision-making.<n>Large language models (LLMs) demonstrate strong reasoning capabilities in closed-world settings, but struggle in open-ended and dynamic environments.<n>Agentic reasoning marks a paradigm shift by reframing LLMs as autonomous agents that plan, act, and learn through continual interaction.
  arXiv  Detail & Related papers  (2026-01-18T18:58:23Z)
• IntentionVLA: Generalizable and Efficient Embodied Intention Reasoning for Human-Robot Interaction [51.130510883952546]
  Vision-Language-Action (VLA) models leverage pretrained vision-language models (VLMs) to couple perception with robotic control.<n>We propose textbfIntentionVLA, a VLA framework with a curriculum training paradigm and an efficient inference mechanism.<n>Our proposed method first leverages carefully designed reasoning data that combine intention inference, spatial grounding, and compact embodied reasoning.
  arXiv  Detail & Related papers  (2025-10-09T04:49:46Z)
• Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making [51.737762570776006]
  LLM-ACTR is a novel neuro-symbolic architecture that provides human-aligned and versatile decision-making. Our framework extracts and embeds knowledge of ACT-R's internal decision-making process as latent neural representations. Our experiments on novel Design for Manufacturing tasks show both improved task performance as well as improved grounded decision-making capability.
  arXiv  Detail & Related papers  (2024-08-17T11:49:53Z)
• Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planning [52.91457780361305]
  This paper introduces cooperative language-guided inverse plan search (CLIPS) Our agent assists a human by modeling them as a cooperative planner who communicates joint plans to the assistant. We evaluate these capabilities in two cooperative planning domains (Doors, Keys & Gems and VirtualHome)
  arXiv  Detail & Related papers  (2024-02-27T23:06:53Z)
• LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models [23.092480882456048]
  Large Language Models (LLMs) have demonstrated emergent common-sense reasoning and Theory of Mind (ToM) capabilities.<n>This study introduces the LLM-Coordination Benchmark, a novel benchmark for analyzing LLMs in the context of Pure Coordination Settings.
  arXiv  Detail & Related papers  (2023-10-05T21:18:15Z)
• Learning to Advise and Learning from Advice in Cooperative Multi-Agent Reinforcement Learning [10.691232629333413]
  Learning toAdvise and Learning from Advice (LALA) is proposed to improve multi-agent reinforcement learning (MARL) The advisor learns to aggregate decision information in both spatial and temporal domains and coordinated decisions by employing a dualtemporal convolutional neural network with a task-oriented objective function. Experimental results indicate the advantage of LALA over baseline approaches in terms of both learning efficiency and coordination capability.
  arXiv  Detail & Related papers  (2022-05-23T09:56:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.