Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillation

• URL: http://arxiv.org/abs/2605.16241v1
• Date: Fri, 15 May 2026 17:48:25 GMT
• Title: Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillation
• Abstract summary: We introduce textbfVLA-AD, a distillation framework that uses a Vision-Language Model as an offline semantic supervisor.<n>Instead of relying on low-level action imitation, VLA-AD augments teacher-provided 7-DoF action targets with high-level semantic guidance.
• Score: 1.5718245345210466
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Billion-parameter Vision-Language-Action (VLA) policies have recently shown impressive performance in robotic manipulation, yet their size and inference cost remain major obstacles for real-time closed-loop control. We introduce \textbf{VLA-AD}, a distillation framework that uses a Vision-Language Model as an offline semantic supervisor to transfer large VLA teachers into lightweight student policies. Instead of relying only on low-level action imitation, VLA-AD augments teacher-provided 7-DoF action targets with high-level semantic guidance, including task phase anchors and multi-frame operating-direction descriptions. These auxiliary signals are used only during training: at test time, the student policy runs independently, with neither the VLA teacher nor the VLM required. We evaluate VLA-AD on three LIBERO benchmark suites. Using OpenVLA-7B as the teacher, our method produces a 158M-parameter student, yielding a $44\times$ reduction in model size while matching the teacher with only a $0.27\%$ average relative gap. The resulting policy runs at 12.5 Hz on an RTX 4090, achieving a $3.28\times$ inference speedup over OpenVLA-7B. We further show that the same semantic distillation pipeline generalizes to a different $π_{0.5}$-4B teacher, where the student outperforms the teacher on two suites and remains within $0.53\%$ on \texttt{libero\_goal}. Additional analysis indicates that phase-level supervision and multi-frame directional cues make the student less sensitive to noisy teacher actions, such as erroneous high-frequency gripper changes. Overall, VLA-AD demonstrates that offline semantic guidance from VLMs can substantially improve the efficiency, robustness, and deployability of VLA policy distillation.

Related papers

• $β$-OPSD: Deriving with Policy Optimization, Training with Self-Distillation [85.07190808882523]
  On-policy self-distillation is a promising approach to improve reasoning language models.<n>We show that $$-OPSD consistently outperforms vanilla OPSD.
  arXiv  Detail & Related papers  (2026-07-30T17:41:16Z)
• TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM [48.08028534478592]
  Vision-language-action (VLA) models commonly adopt an LLM-centric $V to L to A$ pathway.<n>We introduce TurboVLA, a new VLA paradigm that reformulates the conventional $V to L to A$ pathway as a direct $V + L to A$ mapping.<n>On LIBERO, TurboVLA achieves 97.7% average success with only 0.2B parameters, 31.2 ms inference latency, and 0.9 GB inference VRAM.
  arXiv  Detail & Related papers  (2026-07-29T17:59:58Z)
• ROAD-VLA: Robust Online Adaptation via Self-Distillation for Vision-Language-Action Models [11.1063401404824]
  We propose an advantage-guided self-distillation framework that constructs a teacher directly in action space by perturbing action-token logits with calibrated advantage estimates.<n>Across seven robotic manipulation environments with in-distribution and out-of-distribution shifts, ROADVLA outperforms PPO in nearly all settings.
  arXiv  Detail & Related papers  (2026-06-24T13:17:59Z)
• Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning [30.851590309402436]
  Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks.<n>This incurs roughly an order-of-magnitude inference cost from multi-step diffusion.<n>We propose Visual On-Policy Self-Distillation(Visual-OPSD) to solve this problem.
  arXiv  Detail & Related papers  (2026-06-17T11:59:15Z)
• A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model [112.9420001646428]
  Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often constrained by cost.<n>We present A1, a fully open-source and transparent VLA framework designed for low-cost, high- throughput inference.<n>A1 achieves state-of-the-art success rates while significantly reducing inference cost.
  arXiv  Detail & Related papers  (2026-04-07T10:18:40Z)
• Adaptive Capacity Allocation for Vision Language Action Fine-tuning [30.782665306687992]
  Vision language action models (VLAs) are increasingly used for Physical AI, but deploying a pre-trained VLA model to unseen environments still requires adaptation.<n>We present LoRA-SP, a rank-adaptive fine-tuning method that replaces fixed-rank updates with input- and layer-wise capacity.<n>On four real-robot manipulation tasks collected on an unseen AgileX PiPER arm, LoRA-SP matches or exceeds full fine-tuning with far fewer trainable parameters.
  arXiv  Detail & Related papers  (2026-03-08T01:33:01Z)
• DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Action [62.70893433854428]
  We propose DualVLA, which enhances action performance through carefully designed post-training while still preserving reasoning capability.<n>Experiments show that DualVLA achieves an average success rate of 61.0 in SimplerEnv and an average score of 65.4 across eight competitive multimodal benchmarks.
  arXiv  Detail & Related papers  (2025-11-27T06:03:53Z)
• VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation [67.98620973023709]
  VOLD is a framework to transfer reasoning capabilities from text-only teacher models to VLM student models.<n>We show that VOLD outperforms the baseline model significantly and improves over the state of the art by a margin.
  arXiv  Detail & Related papers  (2025-10-27T16:32:12Z)
• HyperVLA: Efficient Inference in Vision-Language-Action Models via Hypernetworks [28.849312355730405]
  Vision-Language-Action (VLA) models have emerged as a promising approach to learning generalist robotic policies.<n>HyperVLA uses a novel hypernetwork (HN)-based architecture that activates only a small task-specific policy during inference.<n>HyperVLA achieves a similar or even higher success rate for both zero-shot generalization and few-shot adaptation.
  arXiv  Detail & Related papers  (2025-10-06T15:15:38Z)
• InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation [43.83789393525928]
  InstructVLA is an end-to-end vision-language model that preserves the flexible reasoning of large vision-language models (VLMs) while delivering leading manipulation performance.<n>InstructVLA introduces a novel training paradigm, Vision-Language-Action Instruction Tuning (VLA-IT), which employs multimodal training with mixture-of-experts adaptation.<n>On in-domain SimplerEnv tasks, InstructVLA achieves 30.5% improvement over SpatialVLA.
  arXiv  Detail & Related papers  (2025-07-23T13:57:06Z)
• SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics [37.82471658370026]
  Vision-language models (VLMs) pretrained on large-scale multimodal datasets encode rich visual and linguistic knowledge.<n>We present SmolVLA, a small, efficient, and community-driven VLA that drastically reduces both training and inference costs.
  arXiv  Detail & Related papers  (2025-06-02T16:30:19Z)
• Refined Policy Distillation: From VLA Generalists to RL Experts [18.186499704928092]
  We introduce Refined Policy Distillation (RPD), a novel Reinforcement Learning-based policy refinement method.<n>RPD distills and refines Vision-Language-Action Models (VLAs) into compact, high-performing expert policies.<n>Our experimental results show that RPD enables the RL student to learn expert policies that outperform the VLA teacher in both dense and sparse reward settings.
  arXiv  Detail & Related papers  (2025-03-06T12:52:11Z)
• Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success [100.226572152954]
  We present an optimized fine-tuning recipe for vision-language-action models (VLAs)<n>Our recipe boosts OpenVLA's average success rate across four task suites from 76.5% to 97.1% while increasing action generation throughput by 26$times$.<n>In real-world evaluations, our fine-tuning recipe enables OpenVLA to successfully execute dexterous, high-frequency control tasks on a bimanual ALOHA robot.
  arXiv  Detail & Related papers  (2025-02-27T00:30:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.