VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing

• URL: http://arxiv.org/abs/2605.30117v1
• Date: Thu, 28 May 2026 15:50:56 GMT
• Title: VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing
• Abstract summary: Vision-Language-Action (VLA) models transform multimodal knowledge into embodied control.<n>We present VLA-Trace, a progressive diagnostic framework that analyzes VLA models through a unified evidence chain.
• Score: 12.088525050678713
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding how Vision-Language-Action (VLA) models transform multimodal knowledge into embodied control remains an open challenge. We present VLA-Trace, a progressive diagnostic framework that analyzes VLA models through a unified evidence chain from representation dynamics to causal control attribution and behavioral manifestation. It specifically combines cross-modal and checkpoint-drift centered kernel alignment (CKA) to trace representation evolution, attention knockout interventions to identify modality-specific control pathways, and rollout-level behavioral probes to examine grounding, shortcut dependence, and semantic following. Experiments on $π_{0.5}$ and OpenVLA reveal three key findings. First, the two models exhibit distinct modality-specific adaptation dynamics during VLA finetuning. Second, they rely on different multimodal routing strategies and layer-wise dependencies during action decoding. Third, although VLA policies excel at visually grounded trajectory generation, they remain limited in fine-grained semantic following. These findings highlight future directions for representation-preserving adaptation, causal VLA circuits, and compositional semantic control.

Related papers

• Explicit Kinematic Guidance from Analytic Concepts for Vision-Language-Action Models [67.98329043920803]
  Current Vision-Language-Action (VLA) models rely mainly on 2D inputs, neglecting the rich object structural information and commonsense knowledge inherent in the 3D physical world.<n>This deficiency restricts their spatial awareness and adaptability for complex, high-precision manipulation.<n>We construct a Concept Expert module for VLA to build executable Analytic Concepts that represent objects as explicit, programmatic blueprints.
  arXiv  Detail & Related papers  (2026-07-29T06:24:25Z)
• Steering Vision-Language Models with Joint Sparse Autoencoders [2.367986397868328]
  We introduce the Joint Sparse Autoencoder (JSAE) to factorize sequence-pooled vision and language activations into shared, interpretable image/caption-level features.<n>Applying JSAE to vision-language models (VLMs) yields representations that are difficult to use as controllable cross-modal steering directions.
  arXiv  Detail & Related papers  (2026-06-24T10:11:04Z)
• Does Visual Information Play a Decisive Role in Vision-Language-Action Model Driving Behavior? [6.847591547447323]
  Vision-Language-Action (VLA) models have demonstrated promising capability in autonomous driving.<n>How current VLA-based driving behavior is grounded in visual information remains poorly understood.<n>We introduce a structured multi-level visual framework to analyze visual-behavior dependency in VLA-based driving models systematically.
  arXiv  Detail & Related papers  (2026-05-29T09:18:32Z)
• Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models [22.535916867005955]
  Large Vision-Language Models (LVLMs) exhibit outstanding performance on vision-language tasks but struggle with hallucination problems.<n>We propose Dynamic Multimodal Activation Steering, a training-free approach for hallucination mitigation.
  arXiv  Detail & Related papers  (2026-02-25T09:10:00Z)
• Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future [125.92052530850425]
  Vision-Language-Action (VLA) frameworks integrate perception with language-grounded decision making.<n>VLA frameworks offer a pathway toward more interpretable, generalizable, and human-aligned driving policies.<n>This work aims to establish a coherent foundation for advancing human-compatible autonomous driving systems.
  arXiv  Detail & Related papers  (2025-12-18T16:57:44Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• When Alignment Fails: Multimodal Adversarial Attacks on Vision-Language-Action Models [75.16145284285456]
  We introduce VLA-Fool, a comprehensive study of multimodal adversarial robustness in embodied VLA models under both white-box and black-box settings.<n>We develop the first automatically crafted and semantically guided prompting framework.<n> Experiments on the LIBERO benchmark reveal that even minor multimodal perturbations can cause significant behavioral deviations.
  arXiv  Detail & Related papers  (2025-11-20T10:14:32Z)
• DAM: Dual Active Learning with Multimodal Foundation Model for Source-Free Domain Adaptation [53.323488295994395]
  Source-free active domain adaptation (SFADA) enhances knowledge transfer from a source model to an unlabeled target domain using limited manual labels selected via active learning.<n>We propose Dual Active learning with Multimodal (DAM) foundation model, a novel framework that integrates multimodal supervision from a ViL model to complement sparse human annotations.<n>Extensive experiments demonstrate that DAM consistently outperforms existing methods and sets a new state-of-the-art across multiple SFADA benchmarks and active learning strategies.
  arXiv  Detail & Related papers  (2025-09-29T15:06:56Z)
• MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings [75.0617088717528]
  MoCa is a framework for transforming pre-trained VLM backbones into effective bidirectional embedding models.<n>MoCa consistently improves performance across MMEB and ViDoRe-v2 benchmarks, achieving new state-of-the-art results.
  arXiv  Detail & Related papers  (2025-06-29T06:41:00Z)
• Treble Counterfactual VLMs: A Causal Approach to Hallucination [6.3952983618258665]
  VisionLanguage Models (VLMs) have advanced multi-modal tasks like image captioning, visual question answering, and reasoning.<n>They often generate hallucinated outputs inconsistent with the visual context or prompt.<n>Existing studies link hallucination to statistical biases, language priors, and biased feature learning but lack a structured causal understanding.
  arXiv  Detail & Related papers  (2025-03-08T11:13:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.