Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation

• URL: http://arxiv.org/abs/2605.27582v1
• Date: Tue, 26 May 2026 18:52:04 GMT
• Title: Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation
• Abstract summary: Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environments it has never seen.<n>We present Uni-LaViRA, a unified agentic architecture that extends the same insight to four task families and to four heterogeneous real robots.
• Score: 60.07205156194741
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environments it has never seen. The dominant approach has been to scale vision-language-action (VLA) foundation models on ever-larger collections of robot trajectories. This paper argues that, for navigation specifically, generality can be obtained structurally, not only through data scale. The underlying decision structure of navigation reduces to a single Language-Vision-Robot Actions Translation. The language action emits semantic-level directional command and the vision action emits a pixel-level visual target. Both outputs lie inside the natural output manifold of pretrained multimodal large language models (MLLMs), so the task can be reasoned about by an agent rather than learned from robot data. Therefore, we present Uni-LaViRA, a unified agentic architecture that extends the same insight to four task families (VLN-CE, ObjectNav, EQA, and Aerial-VLN) and to four heterogeneous real robots (Wheeled, Quadruped, Humanoid robot, and a self-built UAV) in a zero-shot manner. Two agent-loop mechanisms make this unification practical. TODO List Memory (TDM) rewrites a structured checklist of pending sub-goals at every step, reciting the unfinished items back into the agent's most recent attention window. Second Chance Backtrack (SCB) rolls the robot back to the pre-error state and conditions the agent's next plan on the failed sub-trajectory, turning single-pass navigation into a self-correcting process. With zero training effort, Uni-LaViRA reaches 60.7% SR on VLN-CE R2R, 51.3% on VLN-CE RxR, 77.7% on HM3D-v2, 60.0% on HM3D-OVON, 54.7% on MP3D-EQA, and 40.0% on OpenUAV, matching or even surpassing recent training navigation foundation models that consume millions of samples and thousands of GPU-hours.

Related papers

• HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation [19.550897283368894]
  Vision-and-language navigation (VLN) enables robots to follow instructions in previously unseen environments.<n>Ham-VLN is a decision-coupled, agent-authored memory that equips the robot with a persistent, depth-grounded world graph.<n>Ham-VLN achieves 61.0% Success Rate (SR) on VLN-CE R2R, 52.7% SR on VLN-CE RxR, and 79.7% SR on HM3D-v2 ObjectNav without any training.
  arXiv  Detail & Related papers  (2026-07-31T16:32:38Z)
• Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments [96.23886784364997]
  Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation.<n>We present Qwen-VLA, a unified embodied foundation model that extends Qwen's vision-language modeling stack to continuous action and trajectory generation.
  arXiv  Detail & Related papers  (2026-05-28T17:36:31Z)
• AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation [78.80536515102305]
  Vision-and-Language Navigation (VLN) requires an agent to ground language instructions to its own movement within a visual environment.<n>We propose AwareVLN, a novel framework that equips the navigation model with a self-aware reasoning mechanism.<n>Our approach features two key innovations: (1) a structural reasoning module that fosters spatial and task-oriented self-awareness, and (2) an automatic data engine with progress division for effective training.
  arXiv  Detail & Related papers  (2026-05-21T17:58:26Z)
• Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models [2.5338045371474816]
  Vision-Language-Action (VLA) models combine perception, language, and motor control in a single architecture.<n>We apply activation injection, sparse autoencoders, and linear probes to six models spanning 80M--7B parameters across 394,000+ rollout episodes on four benchmarks.
  arXiv  Detail & Related papers  (2026-03-19T17:59:55Z)
• ImagiNav: Scalable Embodied Navigation via Generative Visual Prediction and Inverse Dynamics [13.382453086651019]
  Vision-Language Navigation has relied on end-to-end policies trained on expensive, embodiment-specific robot data.<n>We propose ImagiNav, a novel modular paradigm that decouples visual planning from robot actuation.<n>ImagiNav demonstrates strong zero-shot transfer to robot navigation without requiring robot demonstrations.
  arXiv  Detail & Related papers  (2026-03-14T08:34:02Z)
• DreamToNav: Generalizable Navigation for Robots via Generative Video Planning [1.964570633684439]
  We present DreamToNav, a novel autonomous robot framework that uses generative video models to enable intuitive, human-in-the-loop control.<n>We evaluate the approach on both a wheeled mobile robot and a quadruped robot in indoor navigation tasks.<n>DreamToNav achieves a success rate of 76.7%, with final goal errors typically within 0.05-0.10 m and trajectory tracking errors below 0.15 m.
  arXiv  Detail & Related papers  (2026-03-06T11:57:10Z)
• AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild [62.47761809929869]
  Vision-language navigation (VLN) requires intelligent agents to navigate environments by interpreting linguistic instructions alongside visual observations.<n>Current VLN research for unmanned aerial vehicles (UAVs) relies on detailed, pre-specified instructions to guide the UAV along predetermined routes.<n>We propose AutoFly, an end-to-end Vision-Language-Action model for autonomous UAV navigation.
  arXiv  Detail & Related papers  (2026-02-10T11:08:07Z)
• OpenNav: Open-World Navigation with Multimodal Large Language Models [8.41361699991122]
  Large language models (LLMs) have demonstrated strong common-sense reasoning abilities, making them promising for robotic navigation and planning tasks.<n>We aim to enable robots to interpret and decompose complex language instructions, ultimately synthesizing a sequence of trajectory points to complete diverse navigation tasks.<n>We validate our system on a Husky robot in both indoor and outdoor scenes, demonstrating its real-world robustness and applicability.
  arXiv  Detail & Related papers  (2025-07-24T02:05:28Z)
• NaVILA: Legged Robot Vision-Language-Action Model for Navigation [60.00462044102051]
  It is non-trivial to translate human language instructions all the way to low-level leg joint actions.<n>We propose NaVILA, a 2-level framework that unifies a Vision-Language-Action model (VLA) with locomotion skills.<n>NaVILA substantially improves previous approaches on existing benchmarks.
  arXiv  Detail & Related papers  (2024-12-05T18:58:17Z)
• Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Models [8.668211481067457]
  Co-NavGPT is a novel framework that integrates a Vision Language Model (VLM) as a global planner.<n>Co-NavGPT aggregates sub-maps from multiple robots with diverse viewpoints into a unified global map.<n>The VLM uses this information to assign frontiers across the robots, facilitating coordinated and efficient exploration.
  arXiv  Detail & Related papers  (2023-10-11T23:17:43Z)
• LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action [76.71101507291473]
  We present a system, LM-Nav, for robotic navigation that enjoys the benefits of training on unannotated large datasets of trajectories. We show that such a system can be constructed entirely out of pre-trained models for navigation (ViNG), image-language association (CLIP), and language modeling (GPT-3), without requiring any fine-tuning or language-annotated robot data.
  arXiv  Detail & Related papers  (2022-07-10T10:41:50Z)
• Sim-to-Real Transfer for Vision-and-Language Navigation [70.86250473583354]
  We study the problem of releasing a robot in a previously unseen environment, and having it follow unconstrained natural language navigation instructions. Recent work on the task of Vision-and-Language Navigation (VLN) has achieved significant progress in simulation. To assess the implications of this work for robotics, we transfer a VLN agent trained in simulation to a physical robot.
  arXiv  Detail & Related papers  (2020-11-07T16:49:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.