Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference

• URL: http://arxiv.org/abs/2607.12659v3
• Date: Mon, 20 Jul 2026 13:00:07 GMT
• Title: Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference
• Abstract summary: Vision-Language-Action (VLA) models have achieved impressive performance on diverse embodied tasks.<n> deploying VLA models on low-power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity.<n>Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception-execution misalignment and long reaction time.<n>We propose Jetson-PI, a method for efficient VLA deployment on onboard devices via Foresight-Aligned Asynchronous Correction.
• Score: 23.424767173051404
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language-Action (VLA) models have achieved impressive performance on diverse embodied tasks. However, deploying VLA models on low-power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity, which leads to substantial inference latency and low control frequency. Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception-execution misalignment and long reaction time. In this paper, we propose Jetson-PI, a method for efficient VLA deployment on onboard devices via Foresight-Aligned Asynchronous Correction. To address misalignment, we train a lightweight future correction module that predicts future environment representation conditioned on committed actions, enabling the action expert to directly predict actions from the future time step. To reduce reaction time, we introduce confidence-based scheduling optimization that adaptively balances VLM and action expert invocations, complemented by system-level accelerations including CUDA graph reuse, GPU-resident intermediate buffering, and flow unrolling. Extensive experiments demonstrate that Jetson-PI achieves 8.66x and 5.41x improvements in control frequency compared with naive PyTorch and vla.cpp on NVIDIA Jetson Orin, while outperforming VLASH by 14.8\% in average success rate on the LIBERO benchmark. The code of our asynchronous algorithm is available on https://github.com/PKU-SEC-Lab/Jetson-PI, and our efficient llama.cpp-based inference engine is available on https://github.com/PKU-SEC-Lab/Jetson-PI-Edge.

Related papers

• A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware [0.14913052010438635]
  Large language models (LLMs) are increasingly used as backends for intelligent web services.<n>This paper presents a controlled measurement study of self-hosted LLM inference across edge and near-edge deployment nodes.
  arXiv  Detail & Related papers  (2026-09-08T06:28:32Z)
• Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference [19.683948028719808]
  Vision-Language-Action (VLA) models exhibit strong generalization for robotic manipulation, yet their high inference latency limits real time deployment.<n>We propose a system level acceleration strategy that reduces computation in both perception and action generation.<n> Experiments on Libero, RobotWin, and Real Robot Platforms demonstrate over 2 times speedup while maintaining high performance.
  arXiv  Detail & Related papers  (2026-07-14T02:48:31Z)
• Prefill/Decode-Aware Evaluation of LLM Inference on Emerging AI Accelerators [1.9955035892352748]
  Large language models (LLMs) are increasingly deployed in latency- and cost-sensitive settings.<n>Recent inference systems decompose execution into Prefill and Decode phases, which exhibit distinct computational characteristics.<n>This paper presents a phase-aware evaluation of LLM inference performance across GPU and emerging AI accelerators using a common model, Llama2-7B.
  arXiv  Detail & Related papers  (2026-06-14T12:23:29Z)
• DIRECT: When and Where Should You Allocate Test-Time Compute in Embodied Planners? [57.585275546688116]
  Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents.<n>We argue that choosing when and where to spend test-time compute is central to bringing frontier performance to the real world.<n>We introduce DIRECT, a routing framework that uses multimodal scene context to allocate compute per prompt.
  arXiv  Detail & Related papers  (2026-06-10T17:58:49Z)
• Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment [8.758783768535805]
  Vision-Language-Action (VLA) models are promising for generalist robot control, but on-robot deployment is bottlenecked by real-time inference under tight cost and energy budgets.<n>We present a systematic analysis for low-cost VLA deployment via model-hardware co-characterization.
  arXiv  Detail & Related papers  (2026-04-27T13:12:16Z)
• FASTER: Rethinking Real-Time Flow VLAs [82.58822112377923]
  Real-time execution is crucial for deploying Vision-Language-Action (VLA) models in the physical world.<n>We show that reaction time follows a uniform distribution determined jointly by the Time to First Action (TTFA) and the execution horizon.<n>We propose Fast Action Sampling for ImmediaTE Reaction (FASTER) to overcome this issue.
  arXiv  Detail & Related papers  (2026-03-19T17:51:37Z)
• AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge [49.66156306240961]
  High latency breaks the control loop, rendering powerful models unsafe for real-time deployment.<n>We propose AsyncVLA, an asynchronous control framework that decouples semantic reasoning from reactive execution.<n>AsyncVLA achieves a 40% higher success rate than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-13T21:31:19Z)
• VLA-RAIL: A Real-Time Asynchronous Inference Linker for VLA Models and Robots [5.308743386891208]
  Vision-Language-Action (VLA) models have achieved remarkable breakthroughs in robotics.<n>The strategies for fusing a queue of successive action chunks have a profound impact on the overall performance of VLA models.<n>Existing methods suffer from jitter, stalling, or even pauses in robotic action execution.<n>This paper introduces VLA-RAIL, a novel framework designed to conduct model inference and robot motion control asynchronously.
  arXiv  Detail & Related papers  (2025-12-31T06:59:42Z)
• VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference [24.248289541718275]
  Asynchronous inference offers a promising solution to achieve continuous and low-latency control.<n>We propose VLASH, a general asynchronous inference framework for Vision-Language-Action models.<n>It delivers smooth, accurate, and fast reaction control without additional overhead or architectural changes.
  arXiv  Detail & Related papers  (2025-11-30T18:59:24Z)
• SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration [70.72227437717467]
  Vision-Language-Action (VLA) models have attracted increasing attention for their strong control capabilities.<n>Their high computational cost and low execution frequency hinder their suitability for real-time tasks such as robotic manipulation and autonomous navigation.<n>We propose SP-VLA, a unified framework that accelerates VLA models by jointly scheduling models and pruning tokens.
  arXiv  Detail & Related papers  (2025-06-15T05:04:17Z)
• Real-Time Execution of Action Chunking Flow Policies [49.1574468325115]
  This paper presents a novel inference-time algorithm that enables asynchronous execution of action interacting systems.<n>It is applicable to any diffusion- or VLA-based systems executing out of the box with no re-training.<n>Results show that RTC is fast, performant, and uniquely robust to inference manipulation.
  arXiv  Detail & Related papers  (2025-06-09T01:01:59Z)
• Design and Prototyping Distributed CNN Inference Acceleration in Edge Computing [85.74517957717363]
  HALP accelerates inference by designing a seamless collaboration among edge devices (EDs) in Edge Computing. Experiments show that the distributed inference HALP achieves 1.7x inference acceleration for VGG-16. It is shown that the model selection with distributed inference HALP can significantly improve service reliability.
  arXiv  Detail & Related papers  (2022-11-24T19:48:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.