Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment

• URL: http://arxiv.org/abs/2604.24447v1
• Date: Mon, 27 Apr 2026 13:12:16 GMT
• Title: Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment
• Abstract summary: Vision-Language-Action (VLA) models are promising for generalist robot control, but on-robot deployment is bottlenecked by real-time inference under tight cost and energy budgets.<n>We present a systematic analysis for low-cost VLA deployment via model-hardware co-characterization.
• Score: 8.758783768535805
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language-Action (VLA) models are promising for generalist robot control, but on-robot deployment is bottlenecked by real-time inference under tight cost and energy budgets. Most prior evaluations rely on desktop-grade GPUs, obscuring the trade-offs and opportunities offered by heterogeneous edge accelerators (GPUs/XPUs/NPUs). We present a systematic analysis for low-cost VLA deployment via model-hardware co-characterization. First, we build a cross-accelerator leaderboard and evaluate model-hardware pairs under CET (Cost, Energy, Time), showing that right-sized edge devices can be more cost-/energy-efficient than flagship GPUs while meeting control-rate constraints. Second, using in-depth profiling, we uncover a consistent two-phase inference pattern: a compute-bound VLM backbone followed by a memory-bound Action Expert, which induces phase-dependent underutilization and hardware inefficiency. Finally, guided by these insights, we propose DP-Cache and V-AEFusion to reduce diffusion redundancy and enable asynchronous pipeline parallelism, achieving up to 2.9x speedup on GPUs and 6x on edge NPUs with only marginal success degradation. The example leaderboard website is available at: https://vla-leaderboard-01.vercel.app/.

Related papers

• Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines [0.0]
  Edge-deployed vision systems in target recognition, surveillance, autonomous vehicles, and drone domains require hierarchical inference pipelines.<n>Running all models on the GPU creates a serial bottleneck that limits real-time throughput as pipeline stages grow.<n>We present a five-step methodology for zero fallback DLA backbones.
  arXiv  Detail & Related papers  (2026-08-12T08:14:22Z)
• Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inference [40.7778980764007]
  Vision--action (VLA) models have emerged as a key component in embodied AI.<n>We present Deltoris, an algorithm-hardware co-design framework for efficient diffusion-based VLA inference.
  arXiv  Detail & Related papers  (2026-08-05T04:17:03Z)
• Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference [23.424767173051404]
  Vision-Language-Action (VLA) models have achieved impressive performance on diverse embodied tasks.<n> deploying VLA models on low-power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity.<n>Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception-execution misalignment and long reaction time.<n>We propose Jetson-PI, a method for efficient VLA deployment on onboard devices via Foresight-Aligned Asynchronous Correction.
  arXiv  Detail & Related papers  (2026-07-14T11:38:36Z)
• Edge Physical AI Deployment of Vision Transformers on Heterogeneous Edge GPU Targeting Autonomous Vehicles [8.523119912549232]
  This paper presents Heterogeneous Frame Dispatch Scheduling (H-FraDS) for transformer inference on a recent NVIDIA edge GPU.<n>H-FraDS routes frames across the GPU and dual deep learning accelerator (DLA) cores using fixed dispatch ratios to improve utilization under latency and power constraints.<n>Using Swin Transformer for autonomous-driving perception, H-FraDS Balanced Dispatch (1:2) achieves 125.93 FPS, a 2.36x speedup over standalone adapted-DLA execution.
  arXiv  Detail & Related papers  (2026-07-12T22:06:43Z)
• DIRECT: When and Where Should You Allocate Test-Time Compute in Embodied Planners? [57.585275546688116]
  Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents.<n>We argue that choosing when and where to spend test-time compute is central to bringing frontier performance to the real world.<n>We introduce DIRECT, a routing framework that uses multimodal scene context to allocate compute per prompt.
  arXiv  Detail & Related papers  (2026-06-10T17:58:49Z)
• Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving [54.31800246594724]
  We present Fast-dDrive, a block-diffusion VLA that performs bidirectional refinement within semantic units.<n>We show that Fast-dDrive redefines the speed-accuracy frontier for driving agents.
  arXiv  Detail & Related papers  (2026-05-22T02:31:32Z)
• A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model [112.9420001646428]
  Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often constrained by cost.<n>We present A1, a fully open-source and transparent VLA framework designed for low-cost, high- throughput inference.<n>A1 achieves state-of-the-art success rates while significantly reducing inference cost.
  arXiv  Detail & Related papers  (2026-04-07T10:18:40Z)
• LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Robotics [0.6119773373677944]
  We present LiteVLA-Edge, a deployment-oriented VLA pipeline for fully on-device inference on Jetson Orin-class hardware.<n>Our approach combines supervised image-to-action fine-tuning in FP32 with post-training 4-bit GGUF quantization and GPU-accelerated inference.<n>Under our configuration, LiteVLA-Edge achieves a mean end-to-end runtime of 150.5,ms (approximately 6.6,Hz) while operating entirely offline.
  arXiv  Detail & Related papers  (2026-03-03T03:20:52Z)
• Cross-Platform Scaling of Vision-Language-Action Models from Edge to Cloud GPUs [24.657525276719365]
  Vision-Language-Action (VLA) models have emerged as powerful generalist policies for robotic control.<n>This work presents an evaluation of five representative VLA models.<n>We measure accuracy alongside system-level metrics, including latency, throughput, and peak memory usage.
  arXiv  Detail & Related papers  (2025-09-15T00:00:37Z)
• Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving [4.309392302169281]
  Engine-level prefill-decode (PD) disaggregation avoids interference but incurs higher hardware and coordination overhead.<n>PD achieves up to 2.2x higher throughput, 20x lower TTFT, and 2.5x lower TBT than vLLM; outperforms SG by up to 2x; and matches or exceeds disaggregated vLLM.
  arXiv  Detail & Related papers  (2025-07-09T07:27:18Z)
• SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration [70.72227437717467]
  Vision-Language-Action (VLA) models have attracted increasing attention for their strong control capabilities.<n>Their high computational cost and low execution frequency hinder their suitability for real-time tasks such as robotic manipulation and autonomous navigation.<n>We propose SP-VLA, a unified framework that accelerates VLA models by jointly scheduling models and pruning tokens.
  arXiv  Detail & Related papers  (2025-06-15T05:04:17Z)
• FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge [60.000984252907195]
  Auto-regressive (AR) models have recently shown promise in visual generation tasks due to their superior sampling efficiency.<n>Video generation requires a substantially larger number of tokens to produce coherent temporal frames, resulting in significant overhead during the decoding phase.<n>We propose the textbfFastCar framework to accelerate the decode phase for the AR video generation by exploring the temporal redundancy.
  arXiv  Detail & Related papers  (2025-05-17T05:00:39Z)
• One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation [80.71541671907426]
  OneStep Diffusion Policy (OneDP) is a novel approach that distills knowledge from pre-trained diffusion policies into a single-step action generator. OneDP significantly accelerates response times for robotic control tasks.
  arXiv  Detail & Related papers  (2024-10-28T17:54:31Z)
• Turbocharge Speech Understanding with Pilot Inference [0.9699101045941684]
  This paper sets to accelerate modern speech understanding on resource-constrained edge devices. It takes a hybrid approach: to speed up on-device execution; to offload inputs that are beyond the device's capacity. Our prototype, called PASU, is tested on Arm platforms with 6 - 8 cores: it delivers SOTA accuracy; it reduces the end-to-end latency by 2x and reduces the offloading needs by 2x.
  arXiv  Detail & Related papers  (2023-11-22T17:14:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.