VLAgents: A Policy Server for Efficient VLA Inference

• URL: http://arxiv.org/abs/2601.11250v1
• Date: Fri, 16 Jan 2026 12:58:59 GMT
• Title: VLAgents: A Policy Server for Efficient VLA Inference
• Abstract summary: We introduce VLAgents, a modular policy server that abstracts VLA inferencing behind a unified Gymnasium-style protocol.<n>Its communication layer transparently adapts to the context by supporting both zero-copy shared memory for high-speed simulation and compressed streaming for remote hardware.<n>In a benchmark with both local and remote communication, we demonstrate how it outperforms the default policy servers provided by OpenVLA, OpenPi, and LeRobot.
• Score: 10.731074886477515
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid emergence of Vision-Language-Action models (VLAs) has a significant impact on robotics. However, their deployment remains complex due to the fragmented interfaces and the inherent communication latency in distributed setups. To address this, we introduce VLAgents, a modular policy server that abstracts VLA inferencing behind a unified Gymnasium-style protocol. Crucially, its communication layer transparently adapts to the context by supporting both zero-copy shared memory for high-speed simulation and compressed streaming for remote hardware. In this work, we present the architecture of VLAgents and validate it by integrating seven policies -- including OpenVLA and Pi Zero. In a benchmark with both local and remote communication, we further demonstrate how it outperforms the default policy servers provided by OpenVLA, OpenPi, and LeRobot. VLAgents is available at https://github.com/RobotControlStack/vlagents

Related papers

• vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models [1.8512656289778031]
  We present vla, a portable C++ inference runtime built on llama.<n>It is the first ggml-class engine to serve the flow-matching and diffusion VLA inference pattern.<n>On LIBERO-Object, the engine matches a state-of-the-art checkpoint within one episode out of 200, and runs BitVLA at 100% success in 1.3 GiB of memory.
  arXiv  Detail & Related papers  (2026-06-06T10:45:40Z)
• AgentCVR: Active Multi-Agent Cross-Video Reasoning via Script-Simulated Reinforcement Learning [51.50063777258973]
  Cross-Video Reasoning (CVR) has emerged as a critical frontier in multimodal intelligence.<n>We propose AgentCVR, a multi-agent framework that treats CVR as an active evidence-acquisition task.
  arXiv  Detail & Related papers  (2026-05-28T09:09:41Z)
• StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing [0.0]
  Building generalist embodied agents requires integrating perception, language understanding, and action.<n>We present StarVLA, an open-source for Vision-Language-Action research.
  arXiv  Detail & Related papers  (2026-04-06T17:59:21Z)
• LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Robotics [0.6119773373677944]
  We present LiteVLA-Edge, a deployment-oriented VLA pipeline for fully on-device inference on Jetson Orin-class hardware.<n>Our approach combines supervised image-to-action fine-tuning in FP32 with post-training 4-bit GGUF quantization and GPU-accelerated inference.<n>Under our configuration, LiteVLA-Edge achieves a mean end-to-end runtime of 150.5,ms (approximately 6.6,Hz) while operating entirely offline.
  arXiv  Detail & Related papers  (2026-03-03T03:20:52Z)
• Asynchronous Fast-Slow Vision-Language-Action Policies for Whole-Body Robotic Manipulation [10.09057399213028]
  Vision-Language-Action (VLA) systems integrate a Vision-Language Model (VLM) for semantic reasoning with an action expert generating continuous action signals.<n>We introduce a truly asynchronous Fast-Slow VLA framework (DuoCore-FS) to organize the system into a fast pathway for action generation and a slow pathway for rich VLM reasoning.
  arXiv  Detail & Related papers  (2025-12-23T09:28:20Z)
• BLURR: A Boosted Low-Resource Inference for Vision-Language-Action Models [34.57464032562792]
  Vision-language-action (VLA) models enable impressive zero shot manipulation, but their inference stacks are often too heavy for responsive web demos.<n>We present BLURR, a lightweight inference wrapper that can be plugged into existing VLA controllers without retraining or changing model checkpoints.
  arXiv  Detail & Related papers  (2025-12-12T18:30:45Z)
• Towards Deploying VLA without Fine-Tuning: Plug-and-Play Inference-Time VLA Policy Steering via Embodied Evolutionary Diffusion [8.05008302748311]
  We introduce VLA-Pilot, a plug-and-play inference-time policy steering method for zero-shot deployment of pre-trained VLA policies.<n> Experimental results demonstrate that VLA-Pilot substantially boosts the success rates of off-the-shelf pre-trained VLA policies.
  arXiv  Detail & Related papers  (2025-11-18T06:30:52Z)
• PixelVLA: Advancing Pixel-level Understanding in Vision-Language-Action Model [59.32370587806426]
  Vision-Language-Action models (VLAs) are emerging as powerful tools for learning generalizable visuomotor control policies.<n>We introduce PixelVLA, the first VLA model designed to support both pixel-level reasoning and multimodal prompting with text and visual inputs.<n>Our approach is built on a new visuomotor instruction tuning framework that integrates a multiscale pixel-aware encoder with a visual prompting encoder.
  arXiv  Detail & Related papers  (2025-11-03T13:39:37Z)
• HyperVLA: Efficient Inference in Vision-Language-Action Models via Hypernetworks [28.849312355730405]
  Vision-Language-Action (VLA) models have emerged as a promising approach to learning generalist robotic policies.<n>HyperVLA uses a novel hypernetwork (HN)-based architecture that activates only a small task-specific policy during inference.<n>HyperVLA achieves a similar or even higher success rate for both zero-shot generalization and few-shot adaptation.
  arXiv  Detail & Related papers  (2025-10-06T15:15:38Z)
• dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought [66.78110237549087]
  Vision-Language-Action (VLA) models are emerging as a next-generation paradigm for robotics.<n>We introduce dVLA, a diffusion-based VLA that unifies visual perception, language reasoning, and robotic control in a single system.
  arXiv  Detail & Related papers  (2025-09-30T02:36:11Z)
• From Code to Action: Hierarchical Learning of Diffusion-VLM Policies [8.0703783175731]
  Imitation learning for robotic manipulation often suffers from limited generalization and data scarcity.<n>In this work, we introduce a hierarchical framework that leverages code-generating vision-language models (VLMs)<n>We find that this design enables interpretable policy decomposition, improves generalization when compared to flat policies and enables separate evaluation of high-level planning and low-level control.
  arXiv  Detail & Related papers  (2025-09-29T15:22:18Z)
• ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation [62.58034332427291]
  ForceVLA is a novel end-to-end manipulation framework.<n>It treats external force sensing as a first-class modality within VLA systems.
  arXiv  Detail & Related papers  (2025-05-28T09:24:25Z)
• UFO2: The Desktop AgentOS [60.317812905300336]
  UFO2 is a multiagent AgentOS for Windows desktops that elevates into practical, system-level automation.<n>We evaluate UFO2 across over 20 real-world Windows applications, demonstrating substantial improvements in robustness and execution accuracy over prior CUAs.<n>Our results show that deep OS integration unlocks a scalable path toward reliable, user-aligned desktop automation.
  arXiv  Detail & Related papers  (2025-04-20T13:04:43Z)
• OpenVLA: An Open-Source Vision-Language-Action Model [131.74098076670103]
  We introduce OpenVLA, an open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA shows strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate. We release model checkpoints, fine-tuning notebooks, and our PyTorch with built-in support for training VLAs at scale on Open X-Embodiment datasets.
  arXiv  Detail & Related papers  (2024-06-13T15:46:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.