Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models

• URL: http://arxiv.org/abs/2605.13632v1
• Date: Wed, 13 May 2026 14:58:29 GMT
• Title: Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models
• Abstract summary: GTA-VLA(Guide, Think, Act) is an interactive Vision-Language-Action framework.<n>It enables spatially steerable embodied reasoning by allowing users to guide robot policies with explicit visual cues.
• Score: 15.486973209713954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose GTA-VLA(Guide, Think, Act), an interactive Vision-Language-Action (VLA) framework that enables spatially steerable embodied reasoning by allowing users to guide robot policies with explicit visual cues. Existing VLA models learn a direct "Sense-to-Act" mapping from multimodal observations to robot actions. While effective within the training distribution, such tightly coupled policies are brittle under out-of-domain (OOD) shifts and difficult to correct when failures occur. Although recent embodied Chain-of-Thought (CoT) approaches expose intermediate reasoning, they still lack a mechanism for incorporating human spatial guidance, limiting their ability to resolve visual ambiguities or recover from mistakes. To address this gap, our framework allows users to optionally guide the policy with spatial priors, such as affordance points, boxes, and traces, which the subsequent reasoning process can directly condition on. Based on these inputs, the model generates a unified spatial-visual Chain-of-Thought that integrates external guidance with internal task planning, aligning human visual intent with autonomous decision-making. For practical deployment, we further couple the reasoning module with a lightweight reactive action head for efficient action execution. Extensive experiments demonstrate the effectiveness of our approach. On the in-domain SimplerEnv WidowX benchmark, our framework achieves a state-of-the-art 81.2% success rate. Under OOD visual shifts and spatial ambiguities, a single visual interaction substantially improves task success over existing methods, highlighting the value of interactive reasoning for failure recovery in embodied control. Details of the project can be found here: https://signalispupupu.github.io/GTA-VLA_ProjPage/

Related papers

• CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Intervention [48.20065816234311]
  Vision-Language-Action (VLA) models have driven significant progress in robotic manipulation, yet they struggle with the vision-override phenomenon.<n>We propose CofactVLA, a novel causal intervention framework.<n>We show that CofactVLA establishes a new state-of-the-art across diverse simulation benchmarks.
  arXiv  Detail & Related papers  (2026-08-05T02:58:41Z)
• Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance [108.46436073194546]
  We argue that effective planning should be local, visually grounded, internally generated, and directly aligned with action.<n>We propose Afford-VLA, a unified framework that internalizes task-conditioned affordance as an explicit visual planning interface within VLA models.
  arXiv  Detail & Related papers  (2026-05-22T20:43:47Z)
• GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representations [19.347332945780273]
  Vision-Language-Action (VLA) models have shown strong potential for general-purpose robot manipulation by unifying perception and action.<n>We introduce gesture as a parallel instruction modality and propose a Gesture-aware Vision-Language-Action model (GesVLA)<n> Experimental results show that incorporating gesture consistently improves target grounding accuracy and human-robot interaction efficiency.
  arXiv  Detail & Related papers  (2026-05-21T17:57:44Z)
• From Pixels to Tokens: A Systematic Study of Latent Action Supervision for Vision-Language-Action Models [14.109112325287208]
  We study the study of latent action supervision from two perspectives: (i) regularizing the trajectory via image-based latent actions, and (ii) unifying the target space with action-based latent actions.<n>Our results reveal a formulation-task correspondence, whereas action-based latent actions excel at complex motor coordination.
  arXiv  Detail & Related papers  (2026-05-06T09:27:07Z)
• VLA-IAP: Training-Free Visual Token Pruning via Interaction Alignment for Vision-Language-Action Models [27.12266806191131]
  Vision-Language-Action (VLA) models have rapidly advanced embodied intelligence, enabling robots to execute complex, instruction-driven tasks.<n>Current approaches often prune visually sparse yet structurally critical regions that support manipulation, leading to unstable behavior during early task phases.<n>Our proposed textbftraining-free method, VLA-IAP (Interaction-Aligned Pruning), introduces a geometric prior mechanism to preserve structural anchors and a dynamic scheduling strategy.
  arXiv  Detail & Related papers  (2026-03-24T09:33:05Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• ReViP: Reducing False Completion in Vision-Language-Action Models with Vision-Proprioception Rebalance [50.05984919728878]
  We present ReViP, a novel VLA framework with Vision-Proprioception Rebalance to enhance visual grounding and robustness under perturbations.<n>Specifically, we use an external VLM as a task-stage observer to extract real-time task-centric visual cues from visual observations.<n>To evaluate false completion, we propose the first False-Completion Benchmark Suite built on LIBERO with controlled settings such as Object-Drop.
  arXiv  Detail & Related papers  (2026-01-23T11:31:07Z)
• ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models [15.5794433640853]
  Action Chain-of-Thought (ACoT) is a paradigm where the reasoning process itself is formulated as a structured sequence of coarse action intents.<n>We introduce two complementary components: an Explicit Action Reasoner (EAR) and Implicit Action Reasoner (IAR)
  arXiv  Detail & Related papers  (2026-01-16T16:17:06Z)
• Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future [125.92052530850425]
  Vision-Language-Action (VLA) frameworks integrate perception with language-grounded decision making.<n>VLA frameworks offer a pathway toward more interpretable, generalizable, and human-aligned driving policies.<n>This work aims to establish a coherent foundation for advancing human-compatible autonomous driving systems.
  arXiv  Detail & Related papers  (2025-12-18T16:57:44Z)
• IntentionVLA: Generalizable and Efficient Embodied Intention Reasoning for Human-Robot Interaction [51.130510883952546]
  Vision-Language-Action (VLA) models leverage pretrained vision-language models (VLMs) to couple perception with robotic control.<n>We propose textbfIntentionVLA, a VLA framework with a curriculum training paradigm and an efficient inference mechanism.<n>Our proposed method first leverages carefully designed reasoning data that combine intention inference, spatial grounding, and compact embodied reasoning.
  arXiv  Detail & Related papers  (2025-10-09T04:49:46Z)
• Mechanistic interpretability for steering vision-language-action models [0.23371356738437823]
  Vision-Language-Action (VLA) models are a promising path to realizing generalist embodied agents.<n>We introduce the first framework for interpreting and steering VLAs via their internal representations.<n>We introduce a general-purpose activation steering method that modulates behavior in real time, without fine-tuning, reward signals, or environment interaction.
  arXiv  Detail & Related papers  (2025-08-30T03:01:57Z)
• Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing [62.447497430479174]
  Drawing to reason in space is a novel paradigm that enables LVLMs to reason through elementary drawing operations in the visual space.<n>Our model, named VILASR, consistently outperforms existing methods across diverse spatial reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-06-11T17:41:50Z)
• ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models [26.955482205849282]
  Large Vision-Language Models (LVLMs) have recently advanced robotic manipulation by leveraging vision for scene perception and language for instruction following.<n>We propose ManipLVM-R1, a novel reinforcement learning framework that replaces traditional supervision with Reinforcement Learning using Verifiable Rewards (RLVR).
  arXiv  Detail & Related papers  (2025-05-22T10:57:07Z)
• CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models [89.44024245194315]
  We introduce a method that incorporates explicit visual chain-of-thought (CoT) reasoning into vision-language-action models (VLAs)<n>We introduce CoT-VLA, a state-of-the-art 7B VLA that can understand and generate visual and action tokens.<n>Our experimental results demonstrate that CoT-VLA achieves strong performance, outperforming the state-of-the-art VLA model by 17% in real-world manipulation tasks and 6% in simulation benchmarks.
  arXiv  Detail & Related papers  (2025-03-27T22:23:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.