IROS: A Dual-Process Architecture for Real-Time VLM-Based Indoor Navigation

• URL: http://arxiv.org/abs/2601.21506v1
• Date: Thu, 29 Jan 2026 10:25:14 GMT
• Title: IROS: A Dual-Process Architecture for Real-Time VLM-Based Indoor Navigation
• Abstract summary: IROS is a real-time navigation framework that combines VLM-level contextual reasoning with the efficiency of perceptual modules on low-cost, on-device hardware.<n>IROS improves decision accuracy and reduces latency by 66% compared to continuous VLM-based navigation.
• Score: 1.794583385365075
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Indoor mobile robot navigation requires fast responsiveness and robust semantic understanding, yet existing methods struggle to provide both. Classical geometric approaches such as SLAM offer reliable localization but depend on detailed maps and cannot interpret human-targeted cues (e.g., signs, room numbers) essential for indoor reasoning. Vision-Language-Action (VLA) models introduce semantic grounding but remain strictly reactive, basing decisions only on visible frames and failing to anticipate unseen intersections or reason about distant textual cues. Vision-Language Models (VLMs) provide richer contextual inference but suffer from high computational latency, making them unsuitable for real-time operation on embedded platforms. In this work, we present IROS, a real-time navigation framework that combines VLM-level contextual reasoning with the efficiency of lightweight perceptual modules on low-cost, on-device hardware. Inspired by Dual Process Theory, IROS separates fast reflexive decisions (System One) from slow deliberative reasoning (System Two), invoking the VLM only when necessary. Furthermore, by augmenting compact VLMs with spatial and textual cues, IROS delivers robust, human-like navigation with minimal latency. Across five real-world buildings, IROS improves decision accuracy and reduces latency by 66% compared to continuous VLM-based navigation.

Related papers

• Time-Aware Assistive Navigation [17.665122992601788]
  Current language models rarely plan over whether and when to realize a real-time response to a user.<n>We introduce a large-scale benchmark for an egocentric, assistive navigation task in complex outdoor environments.<n>We show that a simple yet effective modification of the model, including direct supervision to predict the underlying reason for each instruction, yields significant performance gains.
  arXiv  Detail & Related papers  (2026-09-04T17:59:47Z)
• What Limits Vision-and-Language Navigation ? [18.423144111320592]
  StereoNav is a robust Vision-Language-Action framework designed to enhance real-world navigation consistency.<n>Experiments on R2R-CE and RxR-CE demonstrate that StereoNav achieves state-of-the-art egocentric RGB performance.
  arXiv  Detail & Related papers  (2026-05-13T10:41:24Z)
• AgentVLN: Towards Agentic Vision-and-Language Navigation [78.739525400071]
  Vision-and-Language Navigation (VLN) requires an embodied agent to ground complex natural-language instructions into long-horizon navigation in unseen environments.<n>In this paper, we propose AgentVLN, a novel and efficient embodied navigation framework that can be deployed on edge computing platforms.
  arXiv  Detail & Related papers  (2026-03-18T12:43:47Z)
• SFCo-Nav: Efficient Zero-Shot Visual Language Navigation via Collaboration of Slow LLM and Fast Attributed Graph Alignment [7.6686624731596895]
  We present SFCo-Nav, an efficient zero-shot visual language navigation framework.<n>It integrates three key modules: 1) a slow LLM-based planner that produces a strategic chain of subgoals, each linked to an imagined object graph; 2) a fast reactive navigator for real-time object graph construction and subgoal execution; and 3) a lightweight asynchronous slow-fast bridge aligns advanced structured, attributed imagined and perceived graphs to estimate navigation confidence.<n>It matches or exceeds prior state-of-the-art zero-shot VLN success rates while cutting total token consumption per trajectory by over 50% and running more than 3.5
  arXiv  Detail & Related papers  (2026-03-02T05:42:12Z)
• AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge [49.66156306240961]
  High latency breaks the control loop, rendering powerful models unsafe for real-time deployment.<n>We propose AsyncVLA, an asynchronous control framework that decouples semantic reasoning from reactive execution.<n>AsyncVLA achieves a 40% higher success rate than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-13T21:31:19Z)
• AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild [62.47761809929869]
  Vision-language navigation (VLN) requires intelligent agents to navigate environments by interpreting linguistic instructions alongside visual observations.<n>Current VLN research for unmanned aerial vehicles (UAVs) relies on detailed, pre-specified instructions to guide the UAV along predetermined routes.<n>We propose AutoFly, an end-to-end Vision-Language-Action model for autonomous UAV navigation.
  arXiv  Detail & Related papers  (2026-02-10T11:08:07Z)
• CoINS: Counterfactual Interactive Navigation via Skill-Aware VLM [13.352875026471445]
  We propose Counterfactual Interactive Navigation via Skill-aware VLM (CoINS)<n>We fine-tune a VLM, named InterNav-VLM, which incorporates skill affordance and concrete constraint parameters into the input context.<n>To execute the generated high-level plans, we develop a comprehensive skill library through reinforcement learning.
  arXiv  Detail & Related papers  (2026-01-07T14:10:46Z)
• Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigation [52.11339614452127]
  Vision-and-Language Navigation (VLN) requires an agent to dynamically explore complex 3D environments following human instructions.<n>Recent research underscores the potential of harnessing large language models (LLMs) for VLN, given their commonsense knowledge and general reasoning capabilities.<n>We propose a novel dual-process thinking framework dubbed R3, integrating LLMs' generalization capabilities with VLN-specific expertise in a zero-shot manner.
  arXiv  Detail & Related papers  (2025-11-18T04:32:00Z)
• Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigation [16.632191523127865]
  Fast-SmartWay is an end-to-end zero-shot VLN-CE framework that eliminates the need for panoramic views and waypoint predictors.<n>Our approach uses only three frontal RGB-D images combined with natural language instructions, enabling MLLMs to directly predict actions.
  arXiv  Detail & Related papers  (2025-11-02T13:21:54Z)
• dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought [66.78110237549087]
  Vision-Language-Action (VLA) models are emerging as a next-generation paradigm for robotics.<n>We introduce dVLA, a diffusion-based VLA that unifies visual perception, language reasoning, and robotic control in a single system.
  arXiv  Detail & Related papers  (2025-09-30T02:36:11Z)
• Edge-Based Multimodal Sensor Data Fusion with Vision Language Models (VLMs) for Real-time Autonomous Vehicle Accident Avoidance [12.513296074529727]
  This paper proposes the Real-time Edge-based Autonomous Co-pilot Trajectory planner (REACT) for autonomous driving.<n>REACT is a V2X-integrated trajectory optimization framework for AD based on a fine-tuned lightweight Vision-Language Model (VLM)<n> evaluated on the DeepAccident benchmark, REACT achieves state-of-the-art performance, a 77% collision rate reduction, a 48.2% Video Panoptic Quality (VPQ), and a 0.57-second inference latency on the Jetson AGX Orin.
  arXiv  Detail & Related papers  (2025-08-01T20:16:04Z)
• iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMs [4.34964016971127]
  iVISPAR is an interactive benchmark designed to evaluate the spatial reasoning capabilities of Vision-Language Models (VLMs) acting as agents.<n>The benchmark supports visual 3D, 2D, and text-based input modalities, enabling comprehensive assessments of VLMs' planning and reasoning skills.<n>Results indicate that while VLMs perform better on 2D tasks compared to 3D or text-based settings, they struggle with complex spatial configurations and consistently fall short of human performance.
  arXiv  Detail & Related papers  (2025-02-05T14:29:01Z)
• NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning [97.88246428240872]
  Vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions.<n>Recent research has highlighted the promising capacity of large language models (LLMs) in VLN by improving navigational reasoning accuracy and interpretability.<n>This paper introduces a novel strategy called Navigational Chain-of-Thought (NavCoT), where we fulfill parameter-efficient in-domain training to enable self-guided navigational decision.
  arXiv  Detail & Related papers  (2024-03-12T07:27:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.