Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models

• URL: http://arxiv.org/abs/2603.19233v1
• Date: Thu, 19 Mar 2026 17:59:55 GMT
• Title: Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models
• Abstract summary: Vision-Language-Action (VLA) models combine perception, language, and motor control in a single architecture.<n>We apply activation injection, sparse autoencoders, and linear probes to six models spanning 80M--7B parameters across 394,000+ rollout episodes on four benchmarks.
• Score: 2.5338045371474816
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models combine perception, language, and motor control in a single architecture, yet how they translate multimodal inputs into actions remains poorly understood. We apply activation injection, sparse autoencoders (SAEs), and linear probes to six models spanning 80M--7B parameters across 394,000+ rollout episodes on four benchmarks. The visual pathway dominates action generation across all architectures: injecting baseline activations into null-prompt episodes recovers near-identical behavior, while cross-task injection steers robots toward source-task positions (99.8\% of X-VLA episodes align with the source trajectory), exposing spatially bound motor programs tied to scene coordinates rather than abstract task representations. Language sensitivity depends on task structure, not model design: when visual context uniquely specifies the task, language is ignored; when multiple goals share a scene, language becomes essential (X-VLA \texttt{libero\_goal}: 94\%$\to$10\% under wrong prompts vs.\ \texttt{libero\_object}: 60--100\% regardless). In all three multi-pathway architectures (\pizhalf{}, SmolVLA, GR00T), expert pathways encode motor programs while VLM pathways encode goal semantics ($2\times$ greater behavioral displacement from expert injection), and subspace injection confirms these occupy separable activation subspaces. Per-token SAE processing is essential for action fidelity on most architectures, though mean-pooling improves fidelity on X-VLA. Contrastive identification recovers 82+ manipulation concepts, and causal ablation reveals sensitivity spanning 28--92\% zero-effect rates independent of representation width. We release \textbf{Action Atlas} (https://action-atlas.com) for interactive exploration of VLA representations across all six models.

Related papers

• Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-Language-Action Manipulation [5.156481604986546]
  We present Pointing-VLA, a typed hidden-state spatial readout built on Embodied-R1.<n>Pointing-VLA achieves SOTA performance on Bridge/WidowX, averaging 72.9% across the evaluated four-task set.
  arXiv  Detail & Related papers  (2026-08-24T11:43:49Z)
• G0.5: One Autoregressive Stream for Robot Reasoning and Action [43.43377463558725]
  We introduce G0.5, a pretrained autoregressive VLA in which a single transformer decoder emits reasoning and action tokens under a single objective.<n>Because reasoning and action share a single set of weights, the pretrained VLM's capabilities carry over to physical behavior.<n>G0.5 surpasses state-of-the-art models across 7 independent regimes.
  arXiv  Detail & Related papers  (2026-08-12T07:26:47Z)
• Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervision [26.793499191581315]
  Cross-embodiment transfer in vision-language-action (VLA) models remains challenging.<n>We present ZR-0, a 2.6 billion parameter end-to-end VLA model that uses dense Embodied Chain-of-Reason (ECoT) supervision to align cross-embodiment representations.
  arXiv  Detail & Related papers  (2026-06-29T16:48:48Z)
• Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments [96.23886784364997]
  Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation.<n>We present Qwen-VLA, a unified embodied foundation model that extends Qwen's vision-language modeling stack to continuous action and trajectory generation.
  arXiv  Detail & Related papers  (2026-05-28T17:36:31Z)
• Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation [60.07205156194741]
  Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environments it has never seen.<n>We present Uni-LaViRA, a unified agentic architecture that extends the same insight to four task families and to four heterogeneous real robots.
  arXiv  Detail & Related papers  (2026-05-26T18:52:04Z)
• GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representations [19.347332945780273]
  Vision-Language-Action (VLA) models have shown strong potential for general-purpose robot manipulation by unifying perception and action.<n>We introduce gesture as a parallel instruction modality and propose a Gesture-aware Vision-Language-Action model (GesVLA)<n> Experimental results show that incorporating gesture consistently improves target grounding accuracy and human-robot interaction efficiency.
  arXiv  Detail & Related papers  (2026-05-21T17:57:44Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation [70.23578202012048]
  Vision-Language Navigation (VLN) presents a unique challenge for Large Vision-Language Models (VLMs) due to their inherent architectural mismatch.<n>We propose TagaVLM (Topology-Aware Global Action reasoning), an end-to-end framework that explicitly injects topological structures into the VLM backbone.<n>To enhance topological node information, an Interleaved Navigation Prompt strengthens node-level visual-text alignment.<n>With the embedded topological graph, the model is capable of global action reasoning, allowing for robust path correction.
  arXiv  Detail & Related papers  (2026-03-03T13:28:07Z)
• Active Zero: Self-Evolving Vision-Language Models through Active Environment Exploration [72.84714132070404]
  We propose a framework that shifts from passive interaction to active exploration of visual environments.<n>Active-Zero employs three co-evolving agents: a Searcher that retrieves images from open-world repositories based on the model's capability frontier.<n>On Qwen2.5-VL-7B-Instruct across 12 benchmarks, Active-Zero 53.97 average accuracy on reasoning tasks (5.7% improvement) and 59.77 on general understanding (3.9% improvement)
  arXiv  Detail & Related papers  (2026-02-11T17:29:17Z)
• ST4VLA: Spatially Guided Training for Vision-Language-Action Models [80.35847468618276]
  Large vision-language models (VLMs) excel at multimodal understanding but fall short when extended to embodied tasks.<n>We introduce ST4VLA, a dual-system Vision-Language-Action framework to align action learning with spatial priors inVLMs.<n> Empirically, ST4VLA achieves substantial improvements over vanilla VLA, with performance increasing from 66.1 -> 84.6 on Google Robot and from 54.7 -> 73.2 on WidowX Robot.
  arXiv  Detail & Related papers  (2026-02-10T18:59:17Z)
• VIPER Strike: Defeating Visual Reasoning CAPTCHAs via Structured Vision-Language Inference [4.830055389040475]
  Visual Reasoning CAPTCHAs (VRCs) combine visual scenes with natural-language queries that demand compositional inference over objects, attributes, and spatial relations.<n>We present ViPer, a unified attack framework that integrates structured multi-object visual perception with adaptive LLM-based reasoning.<n>ViPer achieves up to 93.2% success, approaching human-level performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2026-01-10T07:01:53Z)
• DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoning [35.44151923549777]
  Vision-Language-Action (VLA) models have recently shown impressive generalization and language-guided manipulation capabilities.<n>Their performance degrades on tasks requiring precise spatial reasoning due to limited spatial reasoning inherited from Vision-Language Models (VLMs)<n>We present DepthVLA, a simple yet effective VLA architecture that explicitly incorporates spatial awareness through a pretrained depth prediction module.
  arXiv  Detail & Related papers  (2025-10-15T10:09:00Z)
• LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Tasks [22.011855291417856]
  It remains underexplored how Vision-Language Models (VLMs) recognize spatial relationships and perceive spatial movement.<n>In this work, we introduce a spatial evaluation pipeline and construct a corresponding benchmark.<n>We conduct experiments on multiple state-of-the-art VLMs and observe that there is significant room for improvement.
  arXiv  Detail & Related papers  (2025-07-27T08:31:24Z)
• Robotic Visual Instruction [10.110333805611974]
  We introduce the Robotic Visual Instruction (RoVI) to guide robotic tasks.<n>RoVI encodes spatial-temporal information into human-interpretable visual instructions.<n>We present Visual Instruction Embodied (VIEW) to enable robots to understand RoVI better.
  arXiv  Detail & Related papers  (2025-05-01T17:55:05Z)
• OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature Extraction [95.6266030753644]
  Vision-Language-Action (VLA) models aim to predict robotic actions based on visual observations and language instructions.<n>Existing approaches require fine-tuning pre-trained vision-language models (VLMs) as visual and language features are independently fed into downstream policies.<n>We propose OTTER, a novel VLA architecture that leverages existing alignments through explicit, text-aware visual feature extraction.
  arXiv  Detail & Related papers  (2025-03-05T18:44:48Z)
• Object-and-Action Aware Model for Visual Language Navigation [70.33142095637515]
  Vision-and-Language Navigation (VLN) is unique in that it requires turning relatively general natural-language instructions into robot agent actions. We propose an Object-and-Action Aware Model (OAAM) that processes these two different forms of natural language based instruction separately. This enables each process to match object-centered/action-centered instruction to their own counterpart visual perception/action orientation flexibly.
  arXiv  Detail & Related papers  (2020-07-29T06:32:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.