Acting While Understanding: Asynchronous Semantic-Action Decoupling for Real-Time Vision-Language-Action Models

• URL: http://arxiv.org/abs/2606.15285v1
• Date: Sat, 13 Jun 2026 12:49:42 GMT
• Title: Acting While Understanding: Asynchronous Semantic-Action Decoupling for Real-Time Vision-Language-Action Models
• Abstract summary: Vision-Language-Action models (VLAs) have demonstrated strong task understanding and generalization in robotic manipulation.<n>We propose an asynchronous semantic-action decoupling framework that separates semantic understanding from action generation.<n>We show that the proposed framework achieves up to 35.6 Hz server-side action- module inference throughput and offers a low-intrusion path to high-frequency closed-loop control.
• Score: 26.864668803085888
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action models (VLAs) have demonstrated strong task understanding and generalization in robotic manipulation, yet the high computational cost of full-model inference limits their deployment in low-latency, high-frequency closed-loop control. We propose an asynchronous semantic-action decoupling framework that separates semantic understanding from action generation along the internal semantic-action interface of existing VLAs, without redesigning the vision-language backbone or introducing an external planner. A low-frequency understanding module asynchronously updates reusable semantic conditions, while a high-frequency action module continuously outputs control actions without repeatedly invoking the full model. To mitigate the temporal mismatch between stale semantics and the current execution state, we further introduce historical action conditioning and time-misalignment training, which provide short-horizon execution context and improve feedback control robustness under stale semantic conditions. Experiments on LIBERO with $π_{0.5}$ and UniVLA, together with real-robot deployment using UniVLA, show that the proposed framework achieves up to 35.6 Hz server-side action-module inference throughput and offers a low-intrusion path to high-frequency closed-loop control without running full VLA inference at control rate.

Related papers

• FlashVLA: Streaming Action Decoding for Fast and Asynchronous VLA Inference [13.128933240091419]
  FlashVLA is a streaming action decoding framework for Vision-Language-Action (VLA) models.<n>It substantially improves inference speed while maintaining strong task performance.<n>It can achieve $geq$30,Hz control frequency on a single GPU with smooth asynchronous inference in real-world deployment.
  arXiv  Detail & Related papers  (2026-08-27T17:19:29Z)
• StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models [84.17128030384099]
  StreamPI is a streaming multimodal temporal modeling framework.<n>It equips single-frame VLA with temporal reasoning capability without introducing any additional parameters.<n> Experiments on real-robot tasks spanning memory-dependent and precise perception scenarios, as well as the simulation benchmark LIBERO, demonstrate that StreamPI outperforms pi0.5 across diverse tasks.
  arXiv  Detail & Related papers  (2026-08-26T17:33:19Z)
• FutureRTC: Real-Time Robot Execution with Anticipatory-Conditioned Action Chunking [51.97839311685636]
  FutureRTC is a plug-and-play adaptation framework that predicts execution-time observations and states for asynchronous VLA control without modifying the underlying policy.<n>We show that FutureRTC achieves robustness to inference delays, resulting in smoother trajectories, faster execution, and consistently higher task success rates.
  arXiv  Detail & Related papers  (2026-07-27T05:10:16Z)
• Action ControlNet: A Lightweight Delay-Aware Adapter for Smooth Asynchronous Control in Vision-Language-Action Models [2.1822977353368778]
  Action ControlNet (ACNet) is a lightweight delay-aware adapter for vision-language--action models.<n>ACNet improves robustness under inference delay and yields smoother asynchronous trajectories than direct chunk stitching.
  arXiv  Detail & Related papers  (2026-06-24T15:53:43Z)
• Closed-Loop Neural Activation Control in Vision-Language-Action Models [19.22723477431829]
  We propose a closed-loop framework that replaces static intervention strength with adaptive, time-varying control signals.<n>We instantiate this framework with both PID and reinforcement learning based controllers.<n> Experiments show that residual-STEER achieves more stable concept regulation and a better steering-task success trade-off than fixed-coefficient baselines.
  arXiv  Detail & Related papers  (2026-05-29T18:59:53Z)
• Sentinel-VLA: A Metacognitive VLA Model with Active Status Monitoring for Dynamic Reasoning and Error Recovery [62.75419724651416]
  We introduce textbfSentinel-VLA, a metacognitive VLA model equipped with an active sentinel'' module to monitor real-time execution status.<n>All training data is automatically generated and annotated through our designed pipeline.<n>Real-world experiments demonstrate that Sentinel-VLA boosts the task success rate by over 30% compared to the SOTA model, PI0.
  arXiv  Detail & Related papers  (2026-05-02T02:10:54Z)
• StreamingVLA: Streaming Vision-Language-Action Model with Action Flow Matching and Adaptive Early Observation [30.881585159777714]
  Vision-language-action (VLA) models have demonstrated exceptional performance in natural language-driven perception and control.<n>High computational cost of VLA models poses significant efficiency challenges.<n>We propose enabling VLAs with the ability to asynchronously parallelize across VLA stages in a "streaming" manner.
  arXiv  Detail & Related papers  (2026-03-30T15:23:27Z)
• Enabling Dynamic Tracking in Vision-Language-Action Models via Time-Discrete and Time-Continuous Velocity Feedforward [11.066720921275648]
  Vision-language-action (VLA) models have shown great promise for robot manipulation.<n>Their deployment on rigid industrial robots remains challenging due to the inherent trade-off between compliance and responsiveness.<n>This paper shows the importance of integrating velocity feedforward terms into VLA policies to resolve this trade-off.
  arXiv  Detail & Related papers  (2026-03-17T07:50:00Z)
• AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge [49.66156306240961]
  High latency breaks the control loop, rendering powerful models unsafe for real-time deployment.<n>We propose AsyncVLA, an asynchronous control framework that decouples semantic reasoning from reactive execution.<n>AsyncVLA achieves a 40% higher success rate than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-13T21:31:19Z)
• TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environments [23.799083918923344]
  We introduce Think-in-Control (TIC)-VLA, a latency-aware framework that explicitly models delayed semantic reasoning during action generation.<n>TIC-VLA defines a delayed semantic-control interface that conditions action generation on delayed vision-language semantic states and explicit latency metadata.<n>To support realistic evaluation, we present DynaNav, a physics-accurate, photo-realistic simulation suite for language-guided navigation in dynamic environments.
  arXiv  Detail & Related papers  (2026-02-02T18:47:49Z)
• AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation [21.23747444669735]
  We propose Action-Context-aware Adaptive Computation for VLA models (AC2-VLA)<n>AC2-VLA adaptively performs cognition reuse across timesteps, token pruning, and selective execution of model components within a unified mechanism.<n>Experiments on robotic manipulation benchmarks show that AC2-VLA achieves up to a 1.79times speedup while reducing FLOPs to 29.4% of the dense baseline.
  arXiv  Detail & Related papers  (2026-01-27T14:10:39Z)
• VITA-E: Natural Embodied Interaction with Concurrent Seeing, Hearing, Speaking, and Acting [66.90028121194636]
  Current Vision-Language-Action (VLA) models are often constrained by a rigid, static interaction paradigm.<n>VITA-E is a novel embodied interaction framework designed for both behavioral and nearly real-time interruption.
  arXiv  Detail & Related papers  (2025-10-21T17:59:56Z)
• SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration [70.72227437717467]
  Vision-Language-Action (VLA) models have attracted increasing attention for their strong control capabilities.<n>Their high computational cost and low execution frequency hinder their suitability for real-time tasks such as robotic manipulation and autonomous navigation.<n>We propose SP-VLA, a unified framework that accelerates VLA models by jointly scheduling models and pruning tokens.
  arXiv  Detail & Related papers  (2025-06-15T05:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.