Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think

• URL: http://arxiv.org/abs/2606.20246v2
• Date: Sat, 20 Jun 2026 12:01:11 GMT
• Title: Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think
• Abstract summary: Vision-Language-Action (VLA) models pre-trained on massive video-robot datasets have revolutionized robotic manipulation.<n>Despite being trained on diverse physical trajectories, they exhibit severe layer-wise representational redundancy.<n>We introduce a structural compression pipeline that is entirely training-free.<n>Downstream fine-tuning of this streamlined architecture yields a 40-50% reduction in training time and up to 30% faster real-time inference.
• Score: 43.01990826833761
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language-Action (VLA) models pre-trained on massive video-robot datasets have revolutionized robotic manipulation, yet their multi-billion parameter architectures impose prohibitive computational burdens during downstream fine-tuning and real-time inference. In this work, we reveal a highly non-trivial architectural characteristic of these continuous control foundation policies (e.g., pi_0, GR00T-N1.5): despite being trained on diverse physical trajectories, they exhibit severe layer-wise representational redundancy. To exploit this, we introduce a structural compression pipeline that is entirely training-free, bypassing the need of existing methods to load full-scale models to learn optimized token reductions or dynamic layer selectors. Instead, using only a single forward pass via Centered Kernel Alignment to identify redundant layer features, we remove twin layers to permanently compress the model depth by up to 50% across both the VLM backbone and the continuous control policy head. Downstream fine-tuning of this streamlined architecture yields a dual acceleration benefit: a 40-50% reduction in training time and up to 30% faster real-time inference, while matching or exceeding full-scale base model performance. We comprehensively validate our method across three simulation benchmarks (LIBERO, RoboCasa, SimplerEnv) and 10 diverse real-world manipulation tasks across 4 unique robotic embodiments. These results prove that advanced VLAs require significantly fewer layers than previously assumed, offering a highly compute-efficient paradigm for scalable robot learning.

Related papers

• Neural Network Optimization Reimagined: Decoupled Techniques for Scratch and Fine-Tuning [49.751529745537546]
  We propose DualOpt, a novel approach that decouples optimization techniques specifically tailored for training from scratch.<n>For training from scratch, we introduce real-time layer-wise weight decay, designed to enhance both convergence and generalization.<n>We extend the layer-wise weight decay to dynamically adjust the rollback levels across layers, adapting to the varying demands of different downstream tasks.
  arXiv  Detail & Related papers  (2026-04-21T06:27:18Z)
• DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA [72.9197085473598]
  DIAL is a framework bridging high-level decision making and low-level motor execution through a differentiable latent intent bottleneck.<n>A VLM-based System-2 performs latent world modeling by latent visual foresight within the VLM's native feature space.<n>A lightweight System-1 policy then decodes this predicted intent together with the current observation into precise robot actions.
  arXiv  Detail & Related papers  (2026-03-31T15:02:27Z)
• Disentangled Robot Learning via Separate Forward and Inverse Dynamics Pretraining [28.30092786035367]
  DeFI is a novel framework that Decouples visual Forward and Inverse dynamics pretraining to exploit respective data sources.<n>We introduce the General Forward Dynamics Model (GFDM), pretrained on diverse human and robot videos for future prediction, and the General Inverse Dynamics Model (GIDM), trained via self-supervised learning to infer latent actions from unlabeled video transitions.<n>Experiments on CALVIN ABC-D and SimplerEnv demonstrate state-of-the-art performance, with DeFI achieving an average task length of 4.51 for CALVIN, 51.2% success rate on SimplerEnv-Frac
  arXiv  Detail & Related papers  (2026-03-27T17:20:10Z)
• DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control [16.562259973551786]
  We introduce DiT4DiT, an end-to-end Video-Action Model that couples a video Diffusion Transformer with an action Diffusion Transformer.<n>Instead of relying on reconstructed future frames, DiT4DiT extracts intermediate denoising features from the video generation process.<n>It achieves state-of-the-art results, reaching average success rates of 98.6% on LIBERO and 50.8% on RoboCasa GR1.
  arXiv  Detail & Related papers  (2026-03-11T06:03:53Z)
• DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editing [67.77471070868852]
  DeepGen 1.0 is a lightweight 5B unified model for image generation and editing.<n>It is trained on only 50M samples, surpassing the 80B HunyuanImage by 28% on WISE and the 27B Qwen-Image-Edit by 37% on UniREditBench.<n>By open-sourcing our training code, weights, and datasets, we provide an efficient, high-performance alternative to democratize unified multimodal research.
  arXiv  Detail & Related papers  (2026-02-12T17:44:24Z)
• ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning [31.000965640377128]
  ABot-M0 is a framework that builds a systematic data curation pipeline.<n>It enables end-to-end transformation of heterogeneous raw data into unified, efficient representations.<n>ABot-M0 supports modular perception via a dual-stream mechanism.
  arXiv  Detail & Related papers  (2026-02-11T16:47:01Z)
• Learning to Accelerate Vision-Language-Action Models through Adaptive Visual Token Caching [25.295588774254952]
  We reformulate inference acceleration as a learnable policy optimization problem.<n>We propose a novel framework that integrates a task-aware decision-making process directly into the Vision-Language-Action model.<n>Our method achieves a 1.76x wall-clock inference speedup while simultaneously improving the average success rate.
  arXiv  Detail & Related papers  (2026-01-31T12:12:51Z)
• InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation [77.07565723756119]
  We present InternVLA-A1, a vision-language-action model with dynamic predictive capabilities.<n>We pre-train these models on heterogeneous data sources over real-world robot data, synthetic simulation data, and human videos.<n>We evaluate InternVLA-A1 on 12 real-world robotic tasks and a simulation benchmark.
  arXiv  Detail & Related papers  (2026-01-05T18:54:29Z)
• When Parameter-efficient Tuning Meets General-purpose Vision-language Models [65.19127815275307]
  PETAL revolutionizes the training process by requiring only 0.5% of the total parameters, achieved through a unique mode approximation technique. Our experiments reveal that PETAL not only outperforms current state-of-the-art methods in most scenarios but also surpasses full fine-tuning models in effectiveness.
  arXiv  Detail & Related papers  (2023-12-16T17:13:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.