Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots

• URL: http://arxiv.org/abs/2607.02501v2
• Date: Fri, 03 Jul 2026 02:16:41 GMT
• Title: Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots
• Abstract summary: We present Embodied$.$, a portable runtime for embodied AI models and WAMs.<n>The runtime provides multi-rate execution, latency-first fused inference, and operator and I/O support.<n>We show that Embodied$.$ improves deployment efficiency while preserving high accuracy across diverse embodied model architectures.
• Score: 19.848719701819153
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Embodied AI models now span vision-language-action (VLA) models and world-action models (WAMs), but practical deployment remains fragmented across model-specific Python stacks, backend assumptions, and robot-side glue code, especially on heterogeneous edge devices. Existing inference runtimes are designed mainly for request-response serving and therefore do not satisfy the runtime contract of embodied deployment: multi-rate execution inside closed-loop control, latency-first batch-1 inference on heterogeneous hardware, and extensible embodied interfaces beyond fixed token I/O. We present Embodied$.$cpp, a portable C++ inference runtime for embodied models. Based on an architectural analysis of representative VLA models and WAMs, Embodied$.$cpp captures a shared execution path and organizes it into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters. The runtime provides modular multi-rate execution, latency-first fused inference, and extensible operator and I/O support, enabling deployment across heterogeneous devices, robots, and simulators through one backend abstraction. We evaluate Embodied$.$cpp on two VLA models, HY-VLA and pi0.5, and on a preliminary WAM benchmark using a LingBot-VA Transformer block. The VLA deployments achieve successful closed-loop execution with 100.0% and 91.0% task success rates, respectively. The WAM benchmark reduces block memory from 312.2 MiB to 88.1 MiB. These results show that Embodied$.$cpp improves deployment efficiency while preserving high accuracy across diverse embodied model architectures.

Related papers

• StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models [84.17128030384099]
  StreamPI is a streaming multimodal temporal modeling framework.<n>It equips single-frame VLA with temporal reasoning capability without introducing any additional parameters.<n> Experiments on real-robot tasks spanning memory-dependent and precise perception scenarios, as well as the simulation benchmark LIBERO, demonstrate that StreamPI outperforms pi0.5 across diverse tasks.
  arXiv  Detail & Related papers  (2026-08-26T17:33:19Z)
• RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents [26.4490744619718]
  Vision-Language-Action (VLA) models have attracted growing interest as a scalable approach to robotic manipulation.<n>Existing solutions address these limitations individually through model retraining or environment-specific modules.<n>We present RoboBRIDGE, a modular framework that provides an orchestration layer over five coordinated modules.
  arXiv  Detail & Related papers  (2026-07-30T08:55:45Z)
• vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models [1.8512656289778031]
  We present vla, a portable C++ inference runtime built on llama.<n>It is the first ggml-class engine to serve the flow-matching and diffusion VLA inference pattern.<n>On LIBERO-Object, the engine matches a state-of-the-art checkpoint within one episode out of 200, and runs BitVLA at 100% success in 1.3 GiB of memory.
  arXiv  Detail & Related papers  (2026-06-06T10:45:40Z)
• OxyGen: Unified KV Cache Management for Vision-Language-Action Models under Multi-Task Parallelism [15.325398725305774]
  We propose unified KV cache management, an inference paradigm that treats KV cache as a first-class shared resource across tasks and over time.<n> OxyGen achieves up to 3.7$times$ speedup over isolated execution, delivering over 200 tokens/s language throughput and 70 Hz action frequency simultaneously without action quality degradation.
  arXiv  Detail & Related papers  (2026-03-15T13:23:56Z)
• AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Models [9.608633915316252]
  Vision-Language-Action (VLA) models demonstrate remarkable potential for generalizable robotic manipulation.<n>Current paradigms rely on coarse, high-level task instructions during supervised fine-tuning.<n>We propose method, the first subtask-aware VLA framework integrated with a scalable offline post-training pipeline.
  arXiv  Detail & Related papers  (2026-03-09T15:52:48Z)
• LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Robotics [0.6119773373677944]
  We present LiteVLA-Edge, a deployment-oriented VLA pipeline for fully on-device inference on Jetson Orin-class hardware.<n>Our approach combines supervised image-to-action fine-tuning in FP32 with post-training 4-bit GGUF quantization and GPU-accelerated inference.<n>Under our configuration, LiteVLA-Edge achieves a mean end-to-end runtime of 150.5,ms (approximately 6.6,Hz) while operating entirely offline.
  arXiv  Detail & Related papers  (2026-03-03T03:20:52Z)
• Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures [4.6111834169518735]
  Vision-Language-Action (VLA) models are an emerging class of workloads critical for robotics and embodied AI at the edge.<n>This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms.
  arXiv  Detail & Related papers  (2026-03-01T01:09:55Z)
• HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies [83.41714103649751]
  Development of embodied intelligence models depends on access to high-quality robot demonstration data.<n>We present HiMoE-VLA, a novel vision-language-action framework tailored to handle diverse robotic data with heterogeneity.<n>HiMoE-VLA demonstrates a consistent performance boost over existing VLA baselines, achieving higher accuracy and robust generalizations.
  arXiv  Detail & Related papers  (2025-12-05T13:21:05Z)
• MOBIUS: Big-to-Mobile Universal Instance Segmentation via Multi-modal Bottleneck Fusion and Calibrated Decoder Pruning [91.90342432541138]
  Scaling up model size and training data has advanced foundation models for instance-level perception.<n>High computational cost limits adoption on resource-constrained platforms.<n>We introduce a new benchmark for efficient segmentation on both high-performance computing platforms and mobile devices.
  arXiv  Detail & Related papers  (2025-10-16T18:00:00Z)
• SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration [70.72227437717467]
  Vision-Language-Action (VLA) models have attracted increasing attention for their strong control capabilities.<n>Their high computational cost and low execution frequency hinder their suitability for real-time tasks such as robotic manipulation and autonomous navigation.<n>We propose SP-VLA, a unified framework that accelerates VLA models by jointly scheduling models and pruning tokens.
  arXiv  Detail & Related papers  (2025-06-15T05:04:17Z)
• Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures [68.91874045918112]
  adapter-ALBERT is an efficient model optimization for maximal data reuse across different tasks. We demonstrate the advantage of mapping the model to a heterogeneous on-chip memory architecture by performing simulations on a validated NLP edge accelerator.
  arXiv  Detail & Related papers  (2023-03-25T14:40:59Z)
• SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devices [69.1412199244903]
  We present a multi-tenant runtime for adaptive model execution with integrated MLOps on edge devices, e.g., a camera, a microphone, or IoT sensors. S SensiX++ operates on two fundamental principles - highly modular componentisation to externalise data operations with clear abstractions and document-centric manifestation for system-wide orchestration. We report on the overall throughput and quantified benefits of various automation components of SensiX++ and demonstrate its efficacy to significantly reduce operational complexity and lower the effort to deploy, upgrade, reconfigure and serve embedded models on edge devices.
  arXiv  Detail & Related papers  (2021-09-08T22:06:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.