Language-Grounded Decoupled Action Representation for Robotic Manipulation

• URL: http://arxiv.org/abs/2603.12967v1
• Date: Fri, 13 Mar 2026 13:08:26 GMT
• Title: Language-Grounded Decoupled Action Representation for Robotic Manipulation
• Abstract summary: We propose the Language-Grounded Decoupled Action Representation (LaDA) framework to connect perception and control.<n>LaDA introduces a fine-grained intermediate layer of three interpretable action primitives--translation, rotation, and gripper control--providing explicit semantic structure for low-level actions.<n>It further employs a semantic-guided soft-label contrastive learning objective to align similar action primitives across tasks, enhancing generalization and motion consistency.
• Score: 78.42228162226839
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The heterogeneity between high-level vision-language understanding and low-level action control remains a fundamental challenge in robotic manipulation. Although recent methods have advanced task-specific action alignment, they often struggle to generate robust and accurate actions for novel or semantically related tasks. To address this, we propose the Language-Grounded Decoupled Action Representation (LaDA) framework, which leverages natural language as a semantic bridge to connect perception and control. LaDA introduces a fine-grained intermediate layer of three interpretable action primitives--translation, rotation, and gripper control--providing explicit semantic structure for low-level actions. It further employs a semantic-guided soft-label contrastive learning objective to align similar action primitives across tasks, enhancing generalization and motion consistency. An adaptive weighting strategy, inspired by curriculum learning, dynamically balances contrastive and imitation objectives for stable and effective training. Extensive experiments on simulated benchmarks (LIBERO and MimicGen) and real-world demonstrations validate that LaDA achieves strong performance and generalizes effectively to unseen or related tasks.

Related papers

• Progressively Learning Heterogeneous Skills in a Unified Latent Space [69.64319218245176]
  HetSkills is a novel framework designed to learn heterogeneous skills within a unified latent space for physics-based character control.<n>HetSkills begins by learning a tracking skill that establishes a strong foundation in motion control and creates a shared motion decoder.<n>We introduce motion distillation to ground text-to-motion generation in language semantics and a task-guidance module that dynamically adjusts actions based on high-level language instructions.
  arXiv  Detail & Related papers  (2026-08-24T13:48:45Z)
• GIVE: Grounding Human Gestures in Vision-Language-Action Models [38.010995423052975]
  GIVE (Gesture Intent via Visual-Semantic Enhancement) is an effective approach to enhance pre-trained Vision-Language-Action (VLA) models with human gesture understanding.<n>In real-world HRI experiments, GIVE substantially outperforms the baseline, improving target object recognition accuracy by 40% and overall task success rate by 80%.
  arXiv  Detail & Related papers  (2026-06-11T14:59:38Z)
• Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulation [24.026684861369386]
  Vision-Language-Action (VLA) models have recently shown strong potential for robot learning by following language instructions.<n>It is difficult to describe which exact object to interact with among similar candidates, where to act on the object, or how the target may change during execution.<n>We propose Gaze2Act, a novel VLA framework that leverages human gaze as a dynamic and intuitive intent signal for complex interactive manipulation.
  arXiv  Detail & Related papers  (2026-05-28T17:37:16Z)
• PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations [66.94988600664574]
  We present textbfPRTS (textbfPrimitive textbfReasoning and textbfTasking textbfSystem), a VLA foundation model that reformulates pretraining through Goal-Conditioned Reinforcement Learning.
  arXiv  Detail & Related papers  (2026-04-30T06:14:02Z)
• Grounding Hierarchical Vision-Language-Action Models Through Explicit Language-Action Alignment [4.001668956134368]
  To be transparent, a robot's natural language communication must be consistent with its actions.<n>Existing hierarchical Vision-Language-Action (VLA) models can generate language through chain-of-thought and low-level actions.<n>We propose a novel training framework that grounds hierarchical VLA sub-task descriptions with respect to the visual observation and action space.
  arXiv  Detail & Related papers  (2026-04-07T09:03:12Z)
• KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition [89.7916653471162]
  We introduce a novel kinematics-rich vision-language-action (VLA) task, in which language commands densely encode diverse kinematic attributes.<n>In this setting, task goals remain invariant while execution trajectories must adapt to instruction-level kinematic specifications.<n>We show that KineVLA consistently outperforms strong VLA baselines on kinematics-sensitive benchmarks.
  arXiv  Detail & Related papers  (2026-03-18T09:28:49Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• Language-Guided Grasp Detection with Coarse-to-Fine Learning for Robotic Manipulation [31.386822229629455]
  We propose Language-Guided Grasp Detection (LGGD) with a coarse-to-fine learning paradigm for robotic manipulation.<n>This design enables fine-grained visual-semantic alignment and improves the feasibility of the predicted grasps with respect to task instructions.<n>Experiments on the OCID-VLG and Grasp-Anything++ datasets show that LGGD surpasses existing language-guided grasping methods.
  arXiv  Detail & Related papers  (2025-12-24T09:16:42Z)
• PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attention [92.85371254435074]
  PosA-VLA framework anchors visual attention via pose-conditioned supervision, consistently guiding the model's perception toward task-relevant regions.<n>We show that our method executes embodied tasks with precise and time-efficient behavior across diverse robotic manipulation benchmarks.
  arXiv  Detail & Related papers  (2025-12-03T12:14:29Z)
• Zero-Shot Open-Vocabulary Human Motion Grounding with Test-Time Training [39.7658823121591]
  ZOMG is a framework that segments motion sequences into semantically meaningful sub-actions without requiring any annotations or fine-tuning.<n>ZOMG integrates (1) language semantic partition, which leverages large language models to decompose instructions into ordered sub-action units, and (2) soft masking optimization.<n>Experiments on three motion-language datasets demonstrate state-of-the-art effectiveness and efficiency of motion grounding performance, outperforming prior methods by +8.7% mAP on HumanML3D benchmark.
  arXiv  Detail & Related papers  (2025-11-19T12:11:36Z)
• Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation [70.8381970762877]
  Vision-Language Models (VLMs) have demonstrated remarkable capabilities in semantic reasoning and task planning.<n>We introduce GRACE, a novel framework that grounds VLM-based reasoning through executable analytic concepts.<n>G GRACE provides a unified and interpretable interface between high-level instruction understanding and low-level robot control.
  arXiv  Detail & Related papers  (2025-10-09T09:08:33Z)
• Grounding Language Models with Semantic Digital Twins for Robotic Planning [6.474368392218828]
  We introduce a novel framework that integrates Semantic Digital Twins (SDTs) with Large Language Models (LLMs)<n>The proposed framework effectively combines high-level reasoning with semantic environment understanding, achieving reliable task completion in the face of uncertainty and failure.
  arXiv  Detail & Related papers  (2025-06-19T17:38:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.