Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models

• URL: http://arxiv.org/abs/2608.23478v1
• Date: Mon, 24 Aug 2026 16:42:49 GMT
• Title: Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models
• Abstract summary: Vision-Language-Action (VLA) models can turn multimodal context into robot actions, but their action decoders are still trained largely by behavior cloning.<n>We propose Intention Distillation (INDI), which distills behavior-level intent into the action decoder.
• Score: 15.173629384871013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models can turn multimodal context into robot actions, but their action decoders are still trained largely by behavior cloning. This supervises which motor command was demonstrated while leaving implicit the local objective served by the behavior under the instruction. Future-based supervision enriches action learning with frames, latent observations, trajectories, or motion representations, but these signals capture particular realizations of what may happen rather than the shared semantic objective of the forthcoming behavior. We propose Intention Distillation (INDI), which distills behavior-level intent into the action decoder. During training, a frozen teacher VLM interprets a demonstrated segment from the current observation, instruction, coarse action summary, and corresponding execution video. From its standard inputs, the deployed VLA recovers the resulting multimodal intent representation at an intermediate decoder layer and uses it to organize action prediction together with representations of how the behavior unfolds and what it achieves. On SimplerEnv-Bridge, INDI improves GR00T-N1.7 from 64.3% to 84.7%, and on RoboCasa Kitchen it improves the controlled GR00T-N1.7 baseline from 64.1% to 70.3%, with consistent gains on $π_{0.5}$ across both benchmarks. In real-world tasks, INDI improves average success from 62.0% to 68.7%, with gains of up to 12.0 pp on longer-horizon tasks. Further analyses show that the recovered latent is used by the decoder, captures behavior objective and execution progress, and organizes downstream predictions in an objective-dependent manner. These results show that action decoders benefit from explicitly modeling the semantic objective of the behavior they generate.

Related papers

• LM-X: Explainable Action Modeling with Progress, Event, and Uncertainty Prediction for Generalist Robot Manipulation [28.431160101245837]
  Generalist vision--language--action (VLA) policies learn long-horizon behavior mainly through short-horizon action prediction.<n>We introduce LM-X, which organizes prediction across task, event, and motor scales without claiming anatomical correspondence.
  arXiv  Detail & Related papers  (2026-08-26T13:05:29Z)
• G0.5: One Autoregressive Stream for Robot Reasoning and Action [43.43377463558725]
  We introduce G0.5, a pretrained autoregressive VLA in which a single transformer decoder emits reasoning and action tokens under a single objective.<n>Because reasoning and action share a single set of weights, the pretrained VLM's capabilities carry over to physical behavior.<n>G0.5 surpasses state-of-the-art models across 7 independent regimes.
  arXiv  Detail & Related papers  (2026-08-12T07:26:47Z)
• Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment [42.503300854478795]
  Anchor-Align improves vision-language-action policies on robot demonstrations.<n>It distills layer-wise representations from a frozen VLM copy to prevent this drift.<n>It converts each action target into a discrete motion-direction label and jointly trains language and action prediction.
  arXiv  Detail & Related papers  (2026-07-15T04:13:54Z)
• WALA Learning Executable Latent Actions from Action-Labeled Demonstrations and Action-Free Videos [38.959415159248046]
  WALA is a framework for learning executable latent actions from both action-labeled demonstrations and action-free videos.<n>We show that WALA achieves strong performance on RoboTwin, sets a new state-of-the-art result on RoboCasa with 75.2% average success.
  arXiv  Detail & Related papers  (2026-07-13T11:02:04Z)
• Sparse2Act: Learning Action-Aligned Sparse 3D Representations for Cross-Domain Robot Manipulation [52.00134393320209]
  We introduce Sparse2Act, an observation-action alignment framework for pretraining sparse point-cloud encoders.<n>Masked sparse 3D tokens are trained to organize scene features around the workspace motion paired with the observation.<n>On the LIBERO-10 benchmark, our method achieves 86.9% average success after 500 fine-tuning steps.
  arXiv  Detail & Related papers  (2026-06-10T23:56:01Z)
• GEAR-VLA: Learning Geometry-Aware Action Representations for Generalizable Robotic Manipulation [49.16739604572808]
  Vision-Language-Action (VLA) models achieve strong benchmark performance but struggle in real-world deployment with unseen objects.<n>We argue that this stems from the lack of a unified geometry-aware manipulation representation.<n>We propose GEAR-VLA, a VLA framework for learning unified geometry-aware action representations for generalizable robotic manipulation.
  arXiv  Detail & Related papers  (2026-06-07T09:23:16Z)
• QuoVLA: Quotient Space for Vision-Language-Action Models [51.02329790939691]
  Vision-Language-Action (VLA) models commonly adapt pretrained VLM latents to robot control by mapping visual observations and language instructions to continuous actions.<n>Our textitQuotient Theory for VLA shows that pretrained VLM latents are not action-insufficient but action-sufficient.<n>We propose QuoVLA, a quotient-space framework for VLA that compresses pretrained VLM latents into action-sufficient representations.
  arXiv  Detail & Related papers  (2026-05-24T06:28:53Z)
• From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Model [76.88595728131288]
  Vision-Language-Action (VLA) models often suffer from performance degradation under distribution shifts.<n>We propose textbfBehaviorVLA, a framework that facilitates robust manipulation through the learning of a temporally coherent behavioral representations.<n> Experiments on RoboTwin 2.0, LIBERO, and CALVIN demonstrate state-of-the-art success rates of 58%, 98%, and 4.36 (Avg.Len)
  arXiv  Detail & Related papers  (2026-05-21T16:14:19Z)
• RotVLA: Rotational Latent Action for Vision-Language-Action Model [54.22746299071677]
  We introduce RotVLA, a VLA framework built on a continuous rotational latent action representation.<n>Latent actions are modeled as elements of SO(n), providing continuity, compositionality, and structured geometry aligned with real-world action dynamics.<n>RotVLA consists of a VLM backbone and a flow-matching action head, pretrained on large-scale cross-embodiment robotic datasets and human videos.
  arXiv  Detail & Related papers  (2026-05-13T11:58:02Z)
• Grounded World Model for Semantically Generalizable Planning [94.53923128709965]
  We learn a Grounded World Model (GWM) in a vision-language-aligned latent space.<n>Each proposed action is scored based on how close its future outcome is to the task instruction.<n>On the proposed WISER benchmark, GWM-MPC achieves a 87% success rate on the test set.
  arXiv  Detail & Related papers  (2026-04-13T17:25:41Z)
• AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps [7.710034405765985]
  AIM is an intent-aware unified world action model that bridges this gap via an explicit spatial interface.<n>Built on a pretrained video generation model, AIM jointly models future observations and value maps within a shared mixture-of-transformers architecture.<n>Experiments on RoboTwin 2.0 benchmark show that AIM achieves a 94.0% average success rate, significantly outperforming prior unified world action baselines.
  arXiv  Detail & Related papers  (2026-04-13T07:48:58Z)
• MVP-LAM: Learning Action-Centric Latent Action via Cross-Viewpoint Reconstruction [11.87290433247564]
  We propose textbfMulti-textbfViewtextbfPoint textbfLatent textbfAction textbfModel (textbfMVP-LAM)<n>MVP-LAM learns discrete latent actions that are highly informative about ground-truth actions from time-synchronized multi-view videos.<n>On Bridge V2, MVP-LAM produces more action-centric latent actions, achieving higher mutual information with ground-truth actions and improved action prediction.
  arXiv  Detail & Related papers  (2026-02-03T15:51:25Z)
• Object-Centric Latent Action Learning [70.3173534658611]
  We propose a novel object-centric latent action learning framework that centers on objects rather than pixels.<n>We leverage self-supervised object-centric pretraining to disentangle action-related and distracting dynamics.<n>Our results show that object-centric pretraining mitigates the negative effects of distractors by 50%.
  arXiv  Detail & Related papers  (2025-02-13T11:27:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.