Reflex: Real-Time VLA Control through Streaming Inference

• URL: http://arxiv.org/abs/2607.14695v1
• Date: Thu, 16 Jul 2026 07:56:43 GMT
• Title: Reflex: Real-Time VLA Control through Streaming Inference
• Abstract summary: We present textbfReflex, a framework that enables textitreal-time streaming inference for flow matching policies.<n>On LIBERO and Kinetix benchmarks, Reflex achieves a 2.58$times$ inference speedup and 50Hz stable streaming, reducing reaction latency by up to 54%.
• Score: 12.85847066934547
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Flow matching Vision-Language-Action (VLA) models promise precise continuous control, but their iterative denoising nature introduces fundamental incompatibilities with real-time robotics: global timestep injection invalidates KV-caching, forcing a choice between slow $O(N^2)$ re-computation or mathematically incorrect cache reuse. We present \textbf{Reflex}, a framework that enables \textit{real-time streaming inference} for flow matching policies by exploiting the \textit{Timestep-Invariance Property} -- that perception encoders are functionally independent of the denoising loop. Reflex partitions the attention context into static, sliding, and dynamic regions, enabling $O(1)$ incremental cache updates while preserving full-batch-equivalent attention outputs for fixed inputs. To ensure stability under continuous high-frequency inference, we introduce \textit{AdaRMSNorm}, an adaptive normalization layer that prevents BFloat16 numerical collapse by gating on flow phase. We further maximize throughput through an \textit{async pipeline} that decouples visual encoding from action generation, combined with \textit{operator fusion} that reduces kernel overhead. On LIBERO and Kinetix benchmarks, Reflex achieves a 2.58$\times$ inference speedup and 50Hz stable streaming, reducing reaction latency by up to 54\% and enabling efficient deployment without performance degradation.

Related papers

• FlashVLA: Streaming Action Decoding for Fast and Asynchronous VLA Inference [13.128933240091419]
  FlashVLA is a streaming action decoding framework for Vision-Language-Action (VLA) models.<n>It substantially improves inference speed while maintaining strong task performance.<n>It can achieve $geq$30,Hz control frequency on a single GPU with smooth asynchronous inference in real-world deployment.
  arXiv  Detail & Related papers  (2026-08-27T17:19:29Z)
• Surprise Forcing: What to Remember, When to Skip in Long Video Generation [65.0022589149937]
  Streaming autoregressive diffusion makes minute-scale video synthesis practical, but its bounded context and fixed denoising schedule allocate resources uniformly.<n>We introduce Surprise Forcing, a training-free framework that treats both limitations as online resource-allocation problems.<n> Experiments on VBench, VBench-Long, and VBench-2.0 show that the proposed allocation strategy improves long-horizon consistency and visual quality while retaining real-time streaming throughput.
  arXiv  Detail & Related papers  (2026-07-20T18:37:58Z)
• Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation [15.129362213838974]
  We propose a trace-aware decoding framework with two components.<n>First, Temporal-Spatial Parallel Decoding (TSPD) uses a lightweight temporalspatial controller that consumes per-token trajectory features, including confidence, entropy, and momentum, together with token position, to decide when a token has converged and can be safely fixed.<n>Second, we introduce Confidence Extrapolation (CE), a training-free state-space module that forecasts future logit trends with uncertainty to support proactive decisions.
  arXiv  Detail & Related papers  (2026-05-29T02:29:28Z)
• Test-time Sparsity for Extreme Fast Action Diffusion [15.679362579177267]
  Action diffusion excels at high-fidelity action generation but incurs heavy computational costs owing to its iterative denoising nature.<n>We propose test-time sparsity to tackle this challenge, which aims to accelerate action diffusion by dynamically predicting prunable residual computations for each model forward at test time.
  arXiv  Detail & Related papers  (2026-05-13T10:28:50Z)
• Motion-Aware Caching for Efficient Autoregressive Video Generation [73.27762884730272]
  MotionCache exploits inter-frame differences as a lightweight proxy for pixel-level motion characteristics.<n>Experiments on state-of-the-art models like SkyReels-V2 and MAGI-1 demonstrate that MotionCache achieves significant speedups.
  arXiv  Detail & Related papers  (2026-05-03T05:49:27Z)
• WorldCache: Content-Aware Caching for Accelerated Video World Models [50.7543797435026]
  We present textbfWorldCache, a Perception-Constrained Dynamic CaCache framework.<n>WorldCache improves both when and how to reuse features.<n>On Cosmos-2.5-2B evaluated on PAI-Bench, WorldCache achieves textbf$2.3times$ inference speedup while preserving textbf99.4% of baseline quality.
  arXiv  Detail & Related papers  (2026-03-23T17:59:54Z)
• AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising [15.787466786514164]
  AvatarForcing is a one-step streaming diffusion framework that denoises a fixed local-future window with heterogeneous noise levels.<n>Experiments on standard benchmarks and a new 400-video long-form benchmark show strong visual quality and lip synchronization at 34 ms/frame.
  arXiv  Detail & Related papers  (2026-03-15T11:42:07Z)
• OmniForcing: Unleashing Real-time Joint Audio-Visual Generation [51.031726911410594]
  Recent joint audio-visual diffusion models achieve remarkable generation quality but suffer from high latency.<n>We propose OmniForcing, the first framework to distill an offline, dual-stream bidirectional diffusion model into a high-fidelity streaming autore generator.
  arXiv  Detail & Related papers  (2026-03-12T08:17:36Z)
• HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising [52.237486207964245]
  HiAR is a hierarchical denoising framework that reverses the conventional generation order.<n>It performs causal generation across all blocks at every denoising step, so that each block is always conditioned on context at the same noise level.<n>On VBench (20s generation), HiAR achieves the best overall score and the lowest temporal drift among all compared methods.
  arXiv  Detail & Related papers  (2026-03-09T17:58:16Z)
• Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding [36.74241893088594]
  Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation.<n>Recent works have accelerated inference via KV cache reuse or decoding, but overlook the intrinsic inefficiencies within the block-wise diffusion process.<n>We propose Streaming-dLLM, a training-free framework that streamlines inference across both spatial and temporal dimensions.
  arXiv  Detail & Related papers  (2026-01-25T17:36:04Z)
• Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformers [95.68243351895107]
  We propose a holistic, video-centric paradigm named textbfLocal textbfDiffusion textbfForcing for textbfVideo textbfFrame textbfInterpolation (LDF-VFI)<n>Our framework is built upon an auto-regressive diffusion transformer that models the entire video sequence to ensure long-range temporal coherence.<n>LDF-VFI achieves state-of-the-art performance on challenging long-sequence benchmarks, demonstrating superior per
  arXiv  Detail & Related papers  (2026-01-21T12:58:52Z)
• TIDAL: Temporally Interleaved Diffusion and Action Loop for High-Frequency VLA Control [15.534182843429043]
  Large-scale Vision-Language-Action (VLA) models offer semantic generalization but suffer from high inference latency.<n>We propose TIDAL, a hierarchical framework that decouples semantic reasoning from high-frequency actuation.<n> TIDAL operates as a backbone-agnostic module for diffusion-basedVLAs, using a dual-frequency architecture.
  arXiv  Detail & Related papers  (2026-01-21T12:43:11Z)
• Real-Time Execution of Action Chunking Flow Policies [49.1574468325115]
  This paper presents a novel inference-time algorithm that enables asynchronous execution of action interacting systems.<n>It is applicable to any diffusion- or VLA-based systems executing out of the box with no re-training.<n>Results show that RTC is fast, performant, and uniquely robust to inference manipulation.
  arXiv  Detail & Related papers  (2025-06-09T01:01:59Z)
• StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation [52.56469577812338]
  We introduce StreamDiffusion, a real-time diffusion pipeline for interactive image generation.<n>Existing diffusion models are adept at creating images from text or image prompts, yet they often fall short in real-time interaction.<n>We present a novel approach that transforms the original sequential denoising into the denoising process.
  arXiv  Detail & Related papers  (2023-12-19T18:18:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.