GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization

• URL: http://arxiv.org/abs/2605.12369v1
• Date: Tue, 12 May 2026 16:38:40 GMT
• Title: GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization
• Abstract summary: We introduce GuidedVLA, a framework designed to guide the action generation to focus on task-relevant factors.<n>Our core insight is to treat the action decoder not as a monolithic learner, but as an assembly of functional components.<n>Our results suggest that explicitly guiding action-decoder learning is a promising direction for building more robust and general VLA models.
• Score: 101.37117235471709
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models aim for general robot learning by aligning action as a modality within powerful Vision-Language Models (VLMs). Existing VLAs rely on end-to-end supervision to implicitly enable the action decoding process to learn task-relevant features. However, without explicit guidance, these models often overfit to spurious correlations, such as visual shortcuts or environmental noise, limiting their generalization. In this paper, we introduce GuidedVLA, a framework designed to manually guide the action generation to focus on task-relevant factors. Our core insight is to treat the action decoder not as a monolithic learner, but as an assembly of functional components. Individual attention heads are supervised by manually defined auxiliary signals to capture distinct factors. As an initial study, we instantiate this paradigm with three specialized heads: object grounding, spatial geometry, and temporal skill logic. Across simulation and real-robot experiments, GuidedVLA improves success rates in both in-domain and out-of-domain settings compared to strong VLA baselines. Finally, we show that the quality of these specialized factors correlates positively with task performance and that our mechanism yields decoupled, high-quality features. Our results suggest that explicitly guiding action-decoder learning is a promising direction for building more robust and general VLA models.

Related papers

• Explicit Kinematic Guidance from Analytic Concepts for Vision-Language-Action Models [67.98329043920803]
  Current Vision-Language-Action (VLA) models rely mainly on 2D inputs, neglecting the rich object structural information and commonsense knowledge inherent in the 3D physical world.<n>This deficiency restricts their spatial awareness and adaptability for complex, high-precision manipulation.<n>We construct a Concept Expert module for VLA to build executable Analytic Concepts that represent objects as explicit, programmatic blueprints.
  arXiv  Detail & Related papers  (2026-07-29T06:24:25Z)
• Action with Visual Primitives [36.230909065494345]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for generalist robotic manipulation.<n>We introduce AVP (Action with Visual Primitives), an end-to-end architecture that implements this visual-primitive-centric interface.<n>We show that AVP improves the success rate by 27.61% over pi_0.5 and outperforms other recent methods.
  arXiv  Detail & Related papers  (2026-05-21T08:52:47Z)
• From Pixels to Tokens: A Systematic Study of Latent Action Supervision for Vision-Language-Action Models [14.109112325287208]
  We study the study of latent action supervision from two perspectives: (i) regularizing the trajectory via image-based latent actions, and (ii) unifying the target space with action-based latent actions.<n>Our results reveal a formulation-task correspondence, whereas action-based latent actions excel at complex motor coordination.
  arXiv  Detail & Related papers  (2026-05-06T09:27:07Z)
• ReFineVLA: Multimodal Reasoning-Aware Generalist Robotic Policies via Teacher-Guided Fine-Tuning [10.039310538887833]
  Vision-Language-Action (VLA) models have gained much attention from the research community.<n>Despite their advancements, VLA models often overlook explicit reasoning and learn the functional input-action mappings.<n>We propose ReFineVLA, a multimodal reasoning-aware framework that fine-tunes VLA models with teacher-guided reasons.
  arXiv  Detail & Related papers  (2026-04-20T04:46:20Z)
• OmniGuide: Universal Guidance Fields for Enhancing Generalist Robot Policies [75.40720507604647]
  Vision-language-action(VLA) models have shown great promise as generalist policies for a large range of relatively simple tasks.<n>We propose a flexible framework that improves VLA performance on such tasks by leveraging arbitrary sources of guidance.<n>We show how many kinds of guidance can be naturally expressed as differentiable energy functions with task-specific attractors and repellers located in 3D space.
  arXiv  Detail & Related papers  (2026-03-09T17:18:13Z)
• ActionCodec: What Makes for Good Action Tokenizers [106.78093973045526]
  Vision-Language-Action (VLA) models have demonstrated superior instruction-following and training efficiency.<n>Central to this paradigm is action tokenization, yet its design has primarily focused on reconstruction fidelity.<n>We introduce textbfActionCodec, a high-performance action tokenizer that significantly enhances both training efficiency and VLA performance.
  arXiv  Detail & Related papers  (2026-02-17T07:07:15Z)
• PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attention [92.85371254435074]
  PosA-VLA framework anchors visual attention via pose-conditioned supervision, consistently guiding the model's perception toward task-relevant regions.<n>We show that our method executes embodied tasks with precise and time-efficient behavior across diverse robotic manipulation benchmarks.
  arXiv  Detail & Related papers  (2025-12-03T12:14:29Z)
• CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models [89.44024245194315]
  We introduce a method that incorporates explicit visual chain-of-thought (CoT) reasoning into vision-language-action models (VLAs)<n>We introduce CoT-VLA, a state-of-the-art 7B VLA that can understand and generate visual and action tokens.<n>Our experimental results demonstrate that CoT-VLA achieves strong performance, outperforming the state-of-the-art VLA model by 17% in real-world manipulation tasks and 6% in simulation benchmarks.
  arXiv  Detail & Related papers  (2025-03-27T22:23:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.