UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation

• URL: http://arxiv.org/abs/2607.06537v2
• Date: Sat, 11 Jul 2026 15:47:01 GMT
• Title: UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation
• Abstract summary: We propose UniLM-Nav, a framework for zero-shot open-vocabulary last-mile navigation.<n>UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and base-pose reasoning.<n>We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points.
• Score: 42.39025692205837
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mobile manipulation requires a robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile navigation. Prior methods for last-mile navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints. We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile navigation. UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning, all resolved with a shared multimodal large language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and robot geometry, it infers a manipulation-ready base pose for the robot. We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.

Related papers

• SSMG-Nav: Enhancing Lifelong Object Navigation with Semantic Skeleton Memory Graph [13.132353682972315]
  We introduce SSMG-Nav, a framework for object navigation built on a textitSemantic Memory Graph (SSMG)<n>SSMG consolidates past observations into a spatially aligned, persistent memory anchored by topological keypoints.<n>For each subgraph, a multimodal prompt synthesized from memory guides the VLM to infer a target belief over destinations.
  arXiv  Detail & Related papers  (2026-03-02T12:48:00Z)
• ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigation [53.95797153529148]
  Embodied agents often struggle with efficient navigation because they rely primarily on partial egocentric observations.<n>We introduce ReasonNavi, a human-inspired framework that operationalizes this reason-then-act paradigm by coupling Multimodal Large Language Models (MLLMs) with deterministic planners.
  arXiv  Detail & Related papers  (2026-01-26T19:09:20Z)
• Learning Category-level Last-meter Navigation from RGB Demonstrations of a Single-instance [5.161531917413708]
  Most RGB-based navigation systems only guarantee coarse, meter-level accuracy.<n>This gap prevents manipulation policies from operating within the distribution of their training demonstrations.<n>We introduce an object-centric imitation learning framework for last-meter navigation.
  arXiv  Detail & Related papers  (2025-12-11T23:35:05Z)
• MoMa-Kitchen: A 100K+ Benchmark for Affordance-Grounded Last-Mile Navigation in Mobile Manipulation [46.30498198691935]
  MoMa-Kitchen is a benchmark dataset for training models to learn optimal final navigation positions.<n>Visual data is collected from RGB-D inputs captured by a first-person view camera mounted on the robotic arm.<n>Our approach enables models to learn affordance-based final positioning that accommodates different arm types and platform heights.
  arXiv  Detail & Related papers  (2025-03-14T04:47:38Z)
• TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigation [52.422619828854984]
  We introduce TopV-Nav, an MLLM-based method that directly reasons on the top-view map with sufficient spatial information.<n>To fully unlock the MLLM's spatial reasoning potential in top-view perspective, we propose the Adaptive Visual Prompt Generation (AVPG) method.
  arXiv  Detail & Related papers  (2024-11-25T14:27:55Z)
• SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation [83.4599149936183]
  Existing zero-shot object navigation methods prompt LLM with the text of spatially closed objects. We propose to represent the observed scene with 3D scene graph. We conduct extensive experiments on MP3D, HM3D and RoboTHOR environments, where SG-Nav surpasses previous state-of-the-art zero-shot methods by more than 10% SR on all benchmarks.
  arXiv  Detail & Related papers  (2024-10-10T17:57:19Z)
• Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigation [58.3480730643517]
  We present LGX, a novel algorithm for Language-Driven Zero-Shot Object Goal Navigation (L-ZSON) Our approach makes use of Large Language Models (LLMs) for this task. We achieve state-of-the-art zero-shot object navigation results on RoboTHOR with a success rate (SR) improvement of over 27% over the current baseline.
  arXiv  Detail & Related papers  (2023-03-06T20:19:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.