VLA Foundry: A Unified Framework for Training Vision-Language-Action Models

• URL: http://arxiv.org/abs/2604.19728v1
• Date: Tue, 21 Apr 2026 17:51:51 GMT
• Title: VLA Foundry: A Unified Framework for Training Vision-Language-Action Models
• Abstract summary: We present VLA Foundry, an open-source framework that unifies LLM, VLM, and VLA training in a single stack.<n>VLA Foundry supports both from-scratch training and pretrained backbones from Hugging Face.<n>We evaluate closed-loop policy performance of both models on LBM Eval, an open-data, open-source simulator.
• Score: 11.774960393195052
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present VLA Foundry, an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. Most open-source VLA efforts specialize on the action training stage, often stitching together incompatible pretraining pipelines. VLA Foundry instead provides a shared training stack with end-to-end control, from language pretraining to action-expert fine-tuning. VLA Foundry supports both from-scratch training and pretrained backbones from Hugging Face. To demonstrate the utility of our framework, we train and release two types of models: the first trained fully from scratch through our LLM-->VLM-->VLA pipeline and the second built on the pretrained Qwen3-VL backbone. We evaluate closed-loop policy performance of both models on LBM Eval, an open-data, open-source simulator. We also contribute usability improvements to the simulator and the STEP analysis tools for easier public use. In the nominal evaluation setting, our fully-open from-scratch model is on par with our prior closed-source work and substituting in the Qwen3-VL backbone leads to a strong multi-task table top manipulation policy outperforming our baseline by a wide margin. The VLA Foundry codebase is available at https://github.com/TRI-ML/vla_foundry and all multi-task model weights are released on https://huggingface.co/collections/TRI-ML/vla-foundry. Additional qualitative videos are available on the project website https://tri-ml.github.io/vla_foundry.

Related papers

• CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving [66.77752555295355]
  We present CLEAR, a system that enables closed-loop training using Reinforcement Learning (RL) at scale for E2E-AD.<n>We show that with a simple reward, CLEAR significantly outperforms previous methods and sets new state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-07-03T00:34:50Z)
• ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation [55.947962672433675]
  ChipMATE is the first self-trained multi-agent framework for RTL generation.<n>Inspired by industrial practice, ChipMATE pairs a Verilog agent with a Python reference-model agent that mutually verify each other's outputs.<n>ChipMATE achieves 75.0% and 80.1% pass@1 on VerilogEval V2 with 4B and 9B base models.
  arXiv  Detail & Related papers  (2026-05-13T01:04:21Z)
• EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training [36.75413663296722]
  Vision-Language-Action Models (VLAs) inherit their visual and linguistic capabilities from Vision-Language Models (VLMs)<n>Most VLAs are built from off-the-shelf VLMs that are not adapted to the embodied domain, limiting their downstream performance.<n>We propose EmbodiedMidtrain to bridge the gap between VLMs and VLAs.
  arXiv  Detail & Related papers  (2026-04-21T21:40:58Z)
• StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing [0.0]
  Building generalist embodied agents requires integrating perception, language understanding, and action.<n>We present StarVLA, an open-source for Vision-Language-Action research.
  arXiv  Detail & Related papers  (2026-04-06T17:59:21Z)
• Rethinking the Practicality of Vision-language-action Model: A Comprehensive Benchmark and An Improved Baseline [38.41143967396976]
  Vision-Language-Action (VLA) models have emerged as a generalist robotic agent.<n>Existing VLAs are hindered by excessive parameter scales, prohibitive pre-training requirements, and limited applicability to diverse embodiments.<n>We propose CEBench, a new benchmark spanning diverse embodiments in both simulation and the real world with consideration of domain randomization.
  arXiv  Detail & Related papers  (2026-02-26T06:27:37Z)
• Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resources [36.525767435183845]
  We introduce Open-Qwen2VL, a fully open-source 2B- parameter Multimodal Large Language Model pre-trained efficiently on 29M image-text pairs.<n>The training was conducted on academic level 8xA100-40G at on 5B packed multimodal tokens, which is 0.36% of 1.4T multimodal pre-training tokens of Qwen2-VL.<n>The final instruction-tuned Open-Qwen2VL outperforms partially-open state-of-the-art MLLM Qwen2-VL-2B on various multimodal benchmarks.
  arXiv  Detail & Related papers  (2025-04-01T09:54:00Z)
• Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success [100.226572152954]
  We present an optimized fine-tuning recipe for vision-language-action models (VLAs)<n>Our recipe boosts OpenVLA's average success rate across four task suites from 76.5% to 97.1% while increasing action generation throughput by 26$times$.<n>In real-world evaluations, our fine-tuning recipe enables OpenVLA to successfully execute dexterous, high-frequency control tasks on a bimanual ALOHA robot.
  arXiv  Detail & Related papers  (2025-02-27T00:30:29Z)
• NVLM: Open Frontier-Class Multimodal LLMs [64.00053046838225]
  We introduce NVLM 1.0, a family of frontier-class multimodal large language models (LLMs) that achieve state-of-the-art results on vision-language tasks. We propose a novel architecture that enhances both training efficiency and multimodal reasoning capabilities. We develop production-grade multimodality for the NVLM-1.0 models, enabling them to excel in vision-language tasks.
  arXiv  Detail & Related papers  (2024-09-17T17:59:06Z)
• OpenVLA: An Open-Source Vision-Language-Action Model [131.74098076670103]
  We introduce OpenVLA, an open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA shows strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate. We release model checkpoints, fine-tuning notebooks, and our PyTorch with built-in support for training VLAs at scale on Open X-Embodiment datasets.
  arXiv  Detail & Related papers  (2024-06-13T15:46:55Z)
• VILA: On Pre-training for Visual Language Models [74.08039416548209]
  We study the design options for VLM pre-training through step-by-step controllable comparisons. We build VILA, a Visual Language model family that consistently outperforms the state-of-the-art models.
  arXiv  Detail & Related papers  (2023-12-12T18:58:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.