Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification

• URL: http://arxiv.org/abs/2608.15636v1
• Date: Sun, 16 Aug 2026 08:59:22 GMT
• Title: Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification
• Abstract summary: Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in the field of embodied AI, but their high computational cost and limited predicted action length hinder real-time deployment.<n>We propose SpecVLA, an algorithm-system co-design framework that adaptively balances action length, inference latency, and task reliability.
• Score: 24.0529851878964
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in the field of embodied AI, but their high computational cost and limited predicted action length hinder real-time deployment. Although Dadu-Corki, a dedicated accelerator for efficient embodied AI, has been introduced, it does not exploit the inherent interaction patterns between the robot and its environment, which results in a relatively short predicted action length. We observe that robotic environments naturally alternate between active states-where precise actions are crucial-and inactive states-where actions have limited impact on task success. This insight enables a new scheduling opportunity: long-action-length speculative prediction in inactive states, paired with selective verification in active states. We propose SpecVLA, an algorithm-system co-design framework that adaptively balances action length, inference latency, and task reliability. On the algorithm side, SpecVLA introduces a state-aware VLA inference execution paradigm and a hardware-friendly construction of a smaller verification model (sVLA) using differential residuals and block-wise mixed-precision quantization. On the system side, we develop a heterogeneous architecture consisting of a GPU and a robotic-specific hardware module, along with a speculative dataflow that decouples VLA and sVLA through parallel execution. Comprehensive evaluations on OpenVLA and RDT across LIBERO and ManiSkill benchmarks show that SpecVLA reduces end-to-end latency significantly while preserving task success rate. By enabling long-action-length speculative prediction with timely verification, SpecVLA achieves real-time robotic manipulation with both high efficiency and reliability.

Related papers

• Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inference [40.7778980764007]
  Vision--action (VLA) models have emerged as a key component in embodied AI.<n>We present Deltoris, an algorithm-hardware co-design framework for efficient diffusion-based VLA inference.
  arXiv  Detail & Related papers  (2026-08-05T04:17:03Z)
• VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies [58.65913948991329]
  We present VISUALTHINK-VLA, a visual intermediate-reasoning framework for accurate, low-latency VLA policies.<n>Our bootstrapping philosophy is to guide action with effective visual thinking.<n>It bootstraps action prediction through a compact visual-evidence interface that preserves spatial precision while avoiding decoding overhead.
  arXiv  Detail & Related papers  (2026-05-28T14:36:53Z)
• OFlow: Injecting Object-Aware Temporal Flow Matching for Robust Robotic Manipulation [55.56490813640669]
  VLA models typically act only on the current frame, while future prediction and object-aware reasoning are often learned in separate latent spaces.<n>We propose OFlow, a framework that addresses both limitations by unifying temporal foresight and object-aware reasoning in a shared semantic latent space.<n>Our method forecasts future latents with temporal flow matching, factorizes them into object-aware representations that emphasize physically relevant cues.
  arXiv  Detail & Related papers  (2026-04-20T06:38:01Z)
• Long-Term Memory for VLA-based Agents in Open-World Task Execution [12.869726287795693]
  Vision-Language-Action (VLA) models have demonstrated significant potential for embodied decision-making.<n>Existing frameworks treat planning and execution as decoupled processes.<n>ChemBot integrates an autonomous AI agent with a progress-aware VLA model for hierarchical task decomposition and execution.
  arXiv  Detail & Related papers  (2026-04-17T03:49:41Z)
• RC-NF: Robot-Conditioned Normalizing Flow for Real-Time Anomaly Detection in Robotic Manipulation [68.7948300643741]
  We propose Robot-Conditioned Normalizing Flow (RC-NF), a real-time monitoring model for robotic anomaly detection and intervention.<n>RC-NF decouples the processing of task-aware robot and object states within the normalizing flow.<n>It achieves state-of-the-art performance across all anomaly types compared to previous methods in monitoring robotic tasks.
  arXiv  Detail & Related papers  (2026-03-11T10:14:37Z)
• Mean-Flow based One-Step Vision-Language-Action [15.497933767026568]
  FlowMatching-based Vision-Language-Action (VLA) frameworks have demonstrated remarkable advantages in generating high-frequency action chunks.<n>They are constrained by prolonged generation latency, which stems from inherent iterative sampling requirements and architectural limitations.<n>We propose a Mean-Flow based One-Step VLA approach, which resolves the noise-induced issues in the action generation process.
  arXiv  Detail & Related papers  (2026-03-02T05:30:30Z)
• TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environments [23.799083918923344]
  We introduce Think-in-Control (TIC)-VLA, a latency-aware framework that explicitly models delayed semantic reasoning during action generation.<n>TIC-VLA defines a delayed semantic-control interface that conditions action generation on delayed vision-language semantic states and explicit latency metadata.<n>To support realistic evaluation, we present DynaNav, a physics-accurate, photo-realistic simulation suite for language-guided navigation in dynamic environments.
  arXiv  Detail & Related papers  (2026-02-02T18:47:49Z)
• StreamVLA: Breaking the Reason-Act Cycle via Completion-State Gating [8.031311182554981]
  Long-horizon robotic manipulation requires bridging the gap between high-level planning and low-level control.<n>Current Vision-Language-Action models often entangle these processes, performing redundant multimodal reasoning at every timestep.<n>We present StreamVLA, a dual-system architecture that unifies textual task decomposition, visual goal imagination, and continuous action generation within a single parameter-efficient backbone.
  arXiv  Detail & Related papers  (2026-02-01T08:51:17Z)
• AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation [21.23747444669735]
  We propose Action-Context-aware Adaptive Computation for VLA models (AC2-VLA)<n>AC2-VLA adaptively performs cognition reuse across timesteps, token pruning, and selective execution of model components within a unified mechanism.<n>Experiments on robotic manipulation benchmarks show that AC2-VLA achieves up to a 1.79times speedup while reducing FLOPs to 29.4% of the dense baseline.
  arXiv  Detail & Related papers  (2026-01-27T14:10:39Z)
• FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenization [61.10456021136654]
  We introduce FASTer, a unified framework for efficient and general robot learning.<n>FASTerVQ encodes action chunks as single-channel images, capturing global-temporal dependencies while maintaining a high compression ratio.<n>FASTerVLA builds on this tokenizer with block-wise autoregressive decoding and a lightweight action expert, achieving both faster inference and higher task performance.
  arXiv  Detail & Related papers  (2025-12-04T16:21:38Z)
• dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought [66.78110237549087]
  Vision-Language-Action (VLA) models are emerging as a next-generation paradigm for robotics.<n>We introduce dVLA, a diffusion-based VLA that unifies visual perception, language reasoning, and robotic control in a single system.
  arXiv  Detail & Related papers  (2025-09-30T02:36:11Z)
• SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration [70.72227437717467]
  Vision-Language-Action (VLA) models have attracted increasing attention for their strong control capabilities.<n>Their high computational cost and low execution frequency hinder their suitability for real-time tasks such as robotic manipulation and autonomous navigation.<n>We propose SP-VLA, a unified framework that accelerates VLA models by jointly scheduling models and pruning tokens.
  arXiv  Detail & Related papers  (2025-06-15T05:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.