Mitigating State Aliasing in Vision-Language-Action Models via Inverse Dynamics Learning

• URL: http://arxiv.org/abs/2605.29577v1
• Date: Thu, 28 May 2026 08:22:49 GMT
• Title: Mitigating State Aliasing in Vision-Language-Action Models via Inverse Dynamics Learning
• Abstract summary: We introduce inverse dynamics learning as an auxiliary objective that directly supervises the Vision-Language-Action (VLA) vision encoder.<n>By predicting the action between current and future observations, our objective encourages the encoder to capture fine-grained visual distinctions that determine low-level actions.<n> Experiments on CALVIN ABC-D and SimplerEnv show consistent gains across diverse VLA baselines.
• Score: 32.59795755994378
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Vision-Language-Action (VLA) models have emerged as a promising framework that unifies perception, reasoning, and control for robot manipulation by adapting pretrained vision-language models (VLMs) to action prediction. However, VLM-derived representations are often insensitive to subtle visual distinctions required for low-level control, causing state aliasing between visually similar states that require substantially different actions. Prior VLA studies improve visual understanding by generating visual or reasoning outputs, such as future frames, 2D grounding points or traces, or intermediate spatial reasoning steps, but these objectives typically shape the vision encoder only indirectly through end-to-end prediction and do not explicitly analyze state aliasing in the learned visual feature space. To mitigate state aliasing, we introduce inverse dynamics learning as an auxiliary objective that directly supervises the VLA vision encoder. By predicting the action between current and future observations, our objective encourages the encoder to capture fine-grained visual distinctions that determine low-level actions. We further use pseudo-reversed supervision to expose the encoder to a broader range of action directions and improve generalization under limited robot demonstrations. Our method applies to diverse VLA baselines, uses only standard observation-action pairs without additional annotations, and preserves the original inference pipeline at test time. Experiments on CALVIN ABC-D and SimplerEnv show consistent gains across diverse VLA baselines. Frozen-encoder probing and state-feature alignment analyses further show that our method learns state-discriminative visual representations that reduce state aliasing and better align with robot state changes.

Related papers

• QuoVLA: Quotient Space for Vision-Language-Action Models [51.02329790939691]
  Vision-Language-Action (VLA) models commonly adapt pretrained VLM latents to robot control by mapping visual observations and language instructions to continuous actions.<n>Our textitQuotient Theory for VLA shows that pretrained VLM latents are not action-insufficient but action-sufficient.<n>We propose QuoVLA, a quotient-space framework for VLA that compresses pretrained VLM latents into action-sufficient representations.
  arXiv  Detail & Related papers  (2026-05-24T06:28:53Z)
• AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignment [8.122021272122668]
  AffordVLA is an affordance-enhanced Vision-Language-Action (VLA) framework that internalizes manipulation-centric affordance perception into VLA visual representations.<n>We show that AffordVLA effectively reshapes VLA visual representations while preserving inference efficiency, leading to improved manipulation success rates and training efficiency.
  arXiv  Detail & Related papers  (2026-05-17T16:02:05Z)
• VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models [55.12929235609365]
  Visual backbones of current vision-language-action (VLA) models are predominantly pretrained on 2D image data without explicit 3D geometric supervision.<n>Existing implicit grounding methods partially address this by aligning VLA features with those of 3D-aware foundation models.<n>We propose VEGA, a framework that directly aligns the output of the VLA's visual encoder with spatially-aware features from DINOv2-FiT3D.
  arXiv  Detail & Related papers  (2026-05-11T12:44:26Z)
• Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models [66.96421290733126]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for robotic manipulation.<n>We propose textbfDeepVision-VLA, built on a textbfVision-Language Mixture-of-Transformers (VL-MoT) framework.<n>DeepVision-VLA outperforms prior state-of-the-art methods by 9.0% and 7.5% on simulated and real-world tasks, respectively.
  arXiv  Detail & Related papers  (2026-03-16T17:59:54Z)
• ReViP: Reducing False Completion in Vision-Language-Action Models with Vision-Proprioception Rebalance [50.05984919728878]
  We present ReViP, a novel VLA framework with Vision-Proprioception Rebalance to enhance visual grounding and robustness under perturbations.<n>Specifically, we use an external VLM as a task-stage observer to extract real-time task-centric visual cues from visual observations.<n>To evaluate false completion, we propose the first False-Completion Benchmark Suite built on LIBERO with controlled settings such as Object-Drop.
  arXiv  Detail & Related papers  (2026-01-23T11:31:07Z)
• CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models [89.44024245194315]
  We introduce a method that incorporates explicit visual chain-of-thought (CoT) reasoning into vision-language-action models (VLAs)<n>We introduce CoT-VLA, a state-of-the-art 7B VLA that can understand and generate visual and action tokens.<n>Our experimental results demonstrate that CoT-VLA achieves strong performance, outperforming the state-of-the-art VLA model by 17% in real-world manipulation tasks and 6% in simulation benchmarks.
  arXiv  Detail & Related papers  (2025-03-27T22:23:04Z)
• Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment [57.0121616203175]
  We propose FiSAO, a novel self-alignment method that utilizes the model's own visual encoder as a fine-grained verifier to improve vision-language alignment.<n>By leveraging token-level feedback from the vision encoder, FiSAO significantly improves vision-language alignment, even surpassing traditional preference tuning methods that require additional data.
  arXiv  Detail & Related papers  (2024-10-18T03:34:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.