Enhancing Linguistic Generalization of VLA: Fine-Tuning OpenVLA via Synthetic Instruction Augmentation

• URL: http://arxiv.org/abs/2603.16044v1
• Date: Tue, 17 Mar 2026 01:04:15 GMT
• Title: Enhancing Linguistic Generalization of VLA: Fine-Tuning OpenVLA via Synthetic Instruction Augmentation
• Abstract summary: Generalization remains a core challenge in embodied AI, as robots must adapt to diverse environments.<n>This paper proposes a parameter-efficient fine-tuning strategy to enhance the linguistic generalization of OpenVLA.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Generalization remains a core challenge in embodied AI, as robots must adapt to diverse environments. While OpenVLA represents the State-of-the-Art (SOTA) in Vision-Language-Action models by leveraging large-scale pre-training, its zero-shot performance can be limited when encountering completely new environments. This paper proposes a parameter-efficient fine-tuning strategy to enhance the linguistic generalization of OpenVLA by synthesizing a general instruction set for the Bridge Dataset V2. The paper leverages a Large Language Model (LLM) to generate a rich variety of semantically equivalent but structurally diverse commands for existing trajectories. In this experiment, Low-Rank Adaptation (LoRA) is implemented to fine-tune OpenVLA on augmented pairs, allowing the model to bridge the gap between complex natural language intent and robotic actions. Results demonstrate that the LoRA-enhanced model's robustness, suggesting that enriching the linguistic space of specialized datasets is crucial for embodied agents.

Related papers

• NebulaVLA: A Dual-Frequency Vision-Language-Action Model With Guide Action for Robotic Manipulation [16.57017306622846]
  We present NebulaVLA, an asynchronous dual-frequency architecture that decouples high-level semantic reasoning from low-level action control.<n>To bridge semantic gaps across heterogeneous robots, we introduce GESTURE-7, a unified language-grounded action representation.
  arXiv  Detail & Related papers  (2026-08-17T12:40:23Z)
• Decoupling the Declarative from the Procedural in Vision-Language-Action Models [65.22639791239932]
  Policy trained to clone a behavior from object-specific demonstrations must generalize beyond that object.<n>We propose w$2$VLA, a new Vision-Language-Action model with restructured information flow.<n>Unlike popular, state-of-the-art VLAs, we show that our modular approach successfully decouples knowledge representations.
  arXiv  Detail & Related papers  (2026-06-19T14:43:58Z)
• Robust Zero-Shot Generalization for Open-Vocabulary Action Recognition via Task Arithmetic [45.298314099319775]
  Open Vocabulary Action Recognition (OVAR) enables the recognition of novel actions by leveraging vision-language representations.<n>We propose an alternative paradigm that bypasses target-domain training and recombines knowledge from existing datasets and models.<n>We show that, in out-of-distribution settings, the resulting merged model achieves superior zero-shot generalization to the pre-trained base model.
  arXiv  Detail & Related papers  (2026-06-17T14:16:12Z)
• PrimitiveVLA: Learning Reusable Motion Primitives for Efficient and Generalizable Robotic Manipulation [50.8450025321217]
  Vision-Language-Action (VLA) models offer a promising paradigm for generalist robotic policies.<n>We argue that these bottlenecks stem from the prevailing Direct Instruction-to-Control Mapping.<n>We propose PrimitiveVLA, a framework that shifts this paradigm toward a Primitive-Centric Disassemble & Assemble paradigm.
  arXiv  Detail & Related papers  (2026-05-27T15:41:18Z)
• $M^2$-VLA: Boosting Vision-Language Models for Generalizable Manipulation via Layer Mixture and Meta-Skills [25.17452377052361]
  Current Vision-Language-Action (VLA) models predominantly rely on end-to-end fine-tuning.<n>We propose $M2$-VLA, which demonstrates that a generalized VLM is able to serve as a powerful backbone for robotic manipulation directly.<n>To overcome this, we introduce the Mixture of Layers (MoL) strategy that selectively extracts task-critical information from dense semantic features.
  arXiv  Detail & Related papers  (2026-04-27T08:44:12Z)
• OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RL [1.880672844596704]
  Visual-Language-Action (VLA) models represent a paradigm shift in embodied AI.<n>Existing frameworks struggle with imprecise perception, suboptimal multimodal fusion, and instability in reinforcement learning.<n>We propose OmniVLA-RL, a novel architecture that leverages a Mix-of-Transformers (MoT) design to integrate reasoning, spatial, and action experts.
  arXiv  Detail & Related papers  (2026-04-20T01:36:58Z)
• TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation [70.23578202012048]
  Vision-Language Navigation (VLN) presents a unique challenge for Large Vision-Language Models (VLMs) due to their inherent architectural mismatch.<n>We propose TagaVLM (Topology-Aware Global Action reasoning), an end-to-end framework that explicitly injects topological structures into the VLM backbone.<n>To enhance topological node information, an Interleaved Navigation Prompt strengthens node-level visual-text alignment.<n>With the embedded topological graph, the model is capable of global action reasoning, allowing for robust path correction.
  arXiv  Detail & Related papers  (2026-03-03T13:28:07Z)
• HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies [83.41714103649751]
  Development of embodied intelligence models depends on access to high-quality robot demonstration data.<n>We present HiMoE-VLA, a novel vision-language-action framework tailored to handle diverse robotic data with heterogeneity.<n>HiMoE-VLA demonstrates a consistent performance boost over existing VLA baselines, achieving higher accuracy and robust generalizations.
  arXiv  Detail & Related papers  (2025-12-05T13:21:05Z)
• LLaDA-VLA: Vision Language Diffusion Action Models [23.653152301133925]
  Masked diffusion models, a paradigm distinct from autoregressive models, have begun to demonstrate competitive performance in text generation and multimodal applications.<n>We present LLaDA-VLA, the first Vision-Language-Diffusion-Action model built upon pretrained d-VLMs for robotic manipulation.
  arXiv  Detail & Related papers  (2025-09-08T17:45:40Z)
• Unified Vision-Language-Action Model [86.68814779303429]
  We present UniVLA, a unified and native multimodal VLA model that autoregressively models vision, language, and action signals as discrete token sequences.<n>Our approach sets new state-of-the-art results across several widely used simulation benchmarks, including CALVIN, LIBERO, and Simplenv-Bridge.<n>We further demonstrate its broad applicability on real-world ALOHA manipulation and autonomous driving.
  arXiv  Detail & Related papers  (2025-06-24T17:59:57Z)
• DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control [7.626715427413578]
  Vision-language-action (VLA) models have shown promise for generalizable robot skills.<n>Current VLA models often focus on scaling the vision-language model (VLM) component, while the action space representation remains a critical bottleneck.<n>This paper introduces DexVLA, a novel framework designed to enhance the efficiency and generalization capabilities ofVLAs for complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2025-02-09T11:25:56Z)
• OpenVLA: An Open-Source Vision-Language-Action Model [131.74098076670103]
  We introduce OpenVLA, an open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA shows strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate. We release model checkpoints, fine-tuning notebooks, and our PyTorch with built-in support for training VLAs at scale on Open X-Embodiment datasets.
  arXiv  Detail & Related papers  (2024-06-13T15:46:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.