LiveVLN: Breaking the Stop-and-Go Loop in Vision-Language Navigation

• URL: http://arxiv.org/abs/2604.19536v1
• Date: Tue, 21 Apr 2026 14:55:51 GMT
• Title: LiveVLN: Breaking the Stop-and-Go Loop in Vision-Language Navigation
• Abstract summary: LiveVLN is a training-free framework for continuous embodied navigation.<n>LiveVLN overlaps execution with the processing of newly arrived observations.
• Score: 48.566721335127106
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent navigation systems achieve strong benchmark results, yet real-world deployment often remains visibly stop-and-go. This bottleneck arises because the sense-inference-execution loop is still blocking: after each new observation, the controller must wait for sensing, transmission, and inference before motion can continue. Reducing action-generation cost alone therefore does not remove redundant waiting. To address this issue, we present LiveVLN, a training-free framework for more continuous embodied navigation by augmenting pretrained VLM navigators with multi-step action continuation. Instead of pausing for each full sense-and-inference round, LiveVLN overlaps execution with the processing of newly arrived observations, allowing refreshed future actions to be handed off before the current executable prefix is exhausted. This design keeps actions continuously available during motion, reducing idle waiting and enabling smoother online execution. The framework operates at runtime and can be integrated with compatible pretrained VLM navigators. Across R2R and RxR, LiveVLN preserves benchmark performance while reducing waiting time and improving action availability. In real-world deployments, it cuts average episode waiting time by up to $77.7\%$ and shortens wall-clock episode time by $12.6\%$ on StreamVLN and $19.6\%$ on NaVIDA, yielding more coherent execution during deployment. Code is available at https://github.com/NIneeeeeem/LiveVLN.

Related papers

• FlashVLA: Streaming Action Decoding for Fast and Asynchronous VLA Inference [13.128933240091419]
  FlashVLA is a streaming action decoding framework for Vision-Language-Action (VLA) models.<n>It substantially improves inference speed while maintaining strong task performance.<n>It can achieve $geq$30,Hz control frequency on a single GPU with smooth asynchronous inference in real-world deployment.
  arXiv  Detail & Related papers  (2026-08-27T17:19:29Z)
• MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation [68.89913903899101]
  We propose a novel VLN framework that achieves state-of-the-art performance with real-time inference efficiency (14 FPS)<n>MemVLN utilizes a visual encoder to process continuous observations and a Large Language Model (LLM) to interpret instructions and generate actions.<n> Experiments on VLN-CE show that MemVLN-4B surpasses the baseline Qwen3-VL-4B architecture by 5.8% SR in R2R and 9.7% SR in RxR, while achieving a 7$times$ speedup in inference latency.
  arXiv  Detail & Related papers  (2026-07-26T07:08:30Z)
• Action ControlNet: A Lightweight Delay-Aware Adapter for Smooth Asynchronous Control in Vision-Language-Action Models [2.1822977353368778]
  Action ControlNet (ACNet) is a lightweight delay-aware adapter for vision-language--action models.<n>ACNet improves robustness under inference delay and yields smoother asynchronous trajectories than direct chunk stitching.
  arXiv  Detail & Related papers  (2026-06-24T15:53:43Z)
• MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving [54.57163800903507]
  We present MindVLA-U1, the first unified streaming VLA architecture for autonomous driving.<n>A unified VLM backbone produces AR language tokens and flow-matching continuous action trajectories in a single forward pass over one shared representation.<n>On the long-tail WOD-E2E benchmark, MindVLA-U1 surpasses experienced human drivers for the first time.
  arXiv  Detail & Related papers  (2026-05-12T18:09:42Z)
• DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executors [57.944744187489185]
  We propose DiscreteRTC, which replaces external corrections with native unmasking.<n>DiscreteRTC is simpler to implement with 0 lines of code for async inpainting, faster at inference with only 0.7x computation compared with generating actions from scratch, and better at execution with 50% higher success rate in real-world dynamic pick task compared with flow-based RTC.
  arXiv  Detail & Related papers  (2026-04-27T23:04:03Z)
• StreamingVLA: Streaming Vision-Language-Action Model with Action Flow Matching and Adaptive Early Observation [30.881585159777714]
  Vision-language-action (VLA) models have demonstrated exceptional performance in natural language-driven perception and control.<n>High computational cost of VLA models poses significant efficiency challenges.<n>We propose enabling VLAs with the ability to asynchronously parallelize across VLA stages in a "streaming" manner.
  arXiv  Detail & Related papers  (2026-03-30T15:23:27Z)
• See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation [59.07792608884117]
  We introduce See, Plan, Rewind (SPR), a progress-aware vision-language-action framework.<n>SPR operates through a continuous core cycle, Seeing the current state and upcoming milestone, Planning a trajectory towards the next 2D waypoint, and Rewinding to a recoverable state upon failure.<n>SPR achieves state-of-the-art robustness with the smallest performance drop, surpassing OpenVLA-OFT and UniVLA.
  arXiv  Detail & Related papers  (2026-03-10T07:22:51Z)
• AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge [49.66156306240961]
  High latency breaks the control loop, rendering powerful models unsafe for real-time deployment.<n>We propose AsyncVLA, an asynchronous control framework that decouples semantic reasoning from reactive execution.<n>AsyncVLA achieves a 40% higher success rate than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-13T21:31:19Z)
• VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference [24.248289541718275]
  Asynchronous inference offers a promising solution to achieve continuous and low-latency control.<n>We propose VLASH, a general asynchronous inference framework for Vision-Language-Action models.<n>It delivers smooth, accurate, and fast reaction control without additional overhead or architectural changes.
  arXiv  Detail & Related papers  (2025-11-30T18:59:24Z)
• Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live [30.099614426825834]
  Continuum is a serving system to optimize job completion time for multi-turn agent workloads.<n>By predicting tool call durations in agentic, Continuum selectively pins the KV cache in GPU memory with a time-to-live value based on total turn number.<n>Our evaluation on real-world agentic workloads with Llama-3.1 8B/70B models shows that Continuum significantly improves the average job completion times.
  arXiv  Detail & Related papers  (2025-11-04T03:43:05Z)
• StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling [27.468345201477504]
  Vision-and-Language Navigation (VLN) in real-world settings requires agents to process continuous visual streams and generate actions with low latency grounded in language instructions.<n>We introduce StreamVLN, a streaming VLN framework that employs a hybrid slow-fast context modeling strategy to support multi-modal reasoning over interleaved vision, language and action inputs.<n> Experiments on VLN-CE benchmarks demonstrate state-of-the-art performance with stable low latency, ensuring robustness and efficiency in real-world deployment.
  arXiv  Detail & Related papers  (2025-07-07T17:49:41Z)
• VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning [77.34267241692706]
  Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions.<n>We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions.
  arXiv  Detail & Related papers  (2025-06-20T17:59:59Z)
• Real-Time Execution of Action Chunking Flow Policies [49.1574468325115]
  This paper presents a novel inference-time algorithm that enables asynchronous execution of action interacting systems.<n>It is applicable to any diffusion- or VLA-based systems executing out of the box with no re-training.<n>Results show that RTC is fast, performant, and uniquely robust to inference manipulation.
  arXiv  Detail & Related papers  (2025-06-09T01:01:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.