Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models

• URL: http://arxiv.org/abs/2603.19183v1
• Date: Thu, 19 Mar 2026 17:42:05 GMT
• Title: Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models
• Abstract summary: Vision-Language-Action (VLA) models have emerged as a promising approach for general-purpose robot manipulation.<n>We apply mechanistic interpretability techniques to better understand the inner workings of VLA models.<n>We show that individual SAE features causally influence robot behavior.
• Score: 7.1750939299528795
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision-Language-Action (VLA) models have emerged as a promising approach for general-purpose robot manipulation. However, their generalization is inconsistent: while these models can perform impressively in some settings, fine-tuned variants often fail on novel objects, scenes, and instructions. We apply mechanistic interpretability techniques to better understand the inner workings of VLA models. To probe internal representations, we train Sparse Autoencoders (SAEs) on hidden layer activations of the VLA. SAEs learn a sparse dictionary whose features act as a compact, interpretable basis for the model's computation. We find that the large majority of extracted SAE features correspond to memorized sequences from specific training demonstrations. However, some features correspond to interpretable, general, and steerable motion primitives and semantic properties, offering a promising glimpse toward VLA generalizability. We propose a metric to categorize features according to whether they represent generalizable transferable primitives or episode-specific memorization. We validate these findings through steering experiments on the LIBERO benchmark. We show that individual SAE features causally influence robot behavior. Steering general features induces behaviors consistent with their semantic meaning and can be applied across tasks and scenes. This work provides the first mechanistic evidence that VLAs can learn generalizable features across tasks and scenes. We observe that supervised fine-tuning on small robotics datasets disproportionately amplifies memorization. In contrast, training on larger, more diverse datasets (e.g., DROID) or using knowledge insulation promotes more general features. We provide an open-source codebase and user-friendly interface for activation collection, SAE training, and feature steering. Our project page is located at http://drvla.github.io

Related papers

• DiMaS: Distribution Matching for Steering Vision-Language-Action Models [41.9662194189542]
  Flow-matching vision-language-action (VLA) models have emerged as powerful policies for robotic manipulation.<n> behavioral control is the ability to govern how a robot performs a task by intervening on its internal representations.<n>We propose DiMaS, a Distribution-Matching Steering strategy tailored to flow-matching VLAs.
  arXiv  Detail & Related papers  (2026-07-15T18:39:13Z)
• Decoupling the Declarative from the Procedural in Vision-Language-Action Models [65.22639791239932]
  Policy trained to clone a behavior from object-specific demonstrations must generalize beyond that object.<n>We propose w$2$VLA, a new Vision-Language-Action model with restructured information flow.<n>Unlike popular, state-of-the-art VLAs, we show that our modular approach successfully decouples knowledge representations.
  arXiv  Detail & Related papers  (2026-06-19T14:43:58Z)
• QuoVLA: Quotient Space for Vision-Language-Action Models [51.02329790939691]
  Vision-Language-Action (VLA) models commonly adapt pretrained VLM latents to robot control by mapping visual observations and language instructions to continuous actions.<n>Our textitQuotient Theory for VLA shows that pretrained VLM latents are not action-insufficient but action-sufficient.<n>We propose QuoVLA, a quotient-space framework for VLA that compresses pretrained VLM latents into action-sufficient representations.
  arXiv  Detail & Related papers  (2026-05-24T06:28:53Z)
• From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data [71.22409934108924]
  Human videos are abundant and capture rich interactions, providing diverse semantic and physical cues for real-world manipulation.<n>This survey provides a unified view of how human videos are transformed into effective knowledge for Vision-Language-Action (VLA) models.<n>We highlight three key open challenges in this area: structuring unstructured videos into training-ready episodes, grounding video-derived supervision into robot-executable actions under embodiment and viewpoint heterogeneity, and designing evaluation protocols that better predict real-world deployment performance and transfer efficiency.
  arXiv  Detail & Related papers  (2026-05-18T06:19:16Z)
• GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization [101.37117235471709]
  We introduce GuidedVLA, a framework designed to guide the action generation to focus on task-relevant factors.<n>Our core insight is to treat the action decoder not as a monolithic learner, but as an assembly of functional components.<n>Our results suggest that explicitly guiding action-decoder learning is a promising direction for building more robust and general VLA models.
  arXiv  Detail & Related papers  (2026-05-12T16:38:40Z)
• Observing and Controlling Features in Vision-Language-Action Models [20.193036403223903]
  Vision-Language-Action Models (VLAs) have shown remarkable progress towards embodied intelligence.<n>We introduce and analyze two main concepts: feature-observability and feature-controllability.<n>Our results show that targeted, lightweight interventions can reliably steer a robot's behavior.
  arXiv  Detail & Related papers  (2026-03-05T18:53:50Z)
• UAOR: Uncertainty-aware Observation Reinjection for Vision-Language-Action Models [25.401818528256232]
  Uncertainty-aware Observation Reinjection (UAOR) is an effective, training-free and plug-and-play module for Vision-Language-Action (VLA) models.<n>Our method consistently improves diverse VLA models across simulation and real-world tasks with minimal overhead.
  arXiv  Detail & Related papers  (2026-02-20T06:22:21Z)
• UniVLA: Learning to Act Anywhere with Task-centric Latent Actions [34.42046035740954]
  UniVLA is a new framework for learning cross-embodiment vision-language-action (VLA) policies.<n>We derive task-centric action representations from videos with a latent action model.<n>We obtain state-of-the-art results across multiple manipulation and navigation benchmarks, as well as real-robot deployments.
  arXiv  Detail & Related papers  (2025-05-09T15:11:13Z)
• OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature Extraction [95.6266030753644]
  Vision-Language-Action (VLA) models aim to predict robotic actions based on visual observations and language instructions.<n>Existing approaches require fine-tuning pre-trained vision-language models (VLMs) as visual and language features are independently fed into downstream policies.<n>We propose OTTER, a novel VLA architecture that leverages existing alignments through explicit, text-aware visual feature extraction.
  arXiv  Detail & Related papers  (2025-03-05T18:44:48Z)
• Latent Action Pretraining from Videos [156.88613023078778]
  We introduce Latent Action Pretraining for general Action models (LAPA)<n>LAPA is an unsupervised method for pretraining Vision-Language-Action (VLA) models without ground-truth robot action labels.<n>We propose a method to learn from internet-scale videos that do not have robot action labels.
  arXiv  Detail & Related papers  (2024-10-15T16:28:09Z)
• Visual Affordance Prediction for Guiding Robot Exploration [56.17795036091848]
  We develop an approach for learning visual affordances for guiding robot exploration. We use a Transformer-based model to learn a conditional distribution in the latent embedding space of a VQ-VAE. We show how the trained affordance model can be used for guiding exploration by acting as a goal-sampling distribution, during visual goal-conditioned policy learning in robotic manipulation.
  arXiv  Detail & Related papers  (2023-05-28T17:53:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.