Learning Category-level Last-meter Navigation from RGB Demonstrations of a Single-instance

• URL: http://arxiv.org/abs/2512.11173v2
• Date: Sun, 15 Feb 2026 00:23:23 GMT
• Title: Learning Category-level Last-meter Navigation from RGB Demonstrations of a Single-instance
• Abstract summary: Most RGB-based navigation systems only guarantee coarse, meter-level accuracy.<n>This gap prevents manipulation policies from operating within the distribution of their training demonstrations.<n>We introduce an object-centric imitation learning framework for last-meter navigation.
• Score: 5.161531917413708
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Achieving precise positioning of the mobile manipulator's base is essential for successful manipulation actions that follow. Most of the RGB-based navigation systems only guarantee coarse, meter-level accuracy, making them less suitable for the precise positioning phase of mobile manipulation. This gap prevents manipulation policies from operating within the distribution of their training demonstrations, resulting in frequent execution failures. We address this gap by introducing an object-centric imitation learning framework for last-meter navigation, enabling a quadruped mobile manipulator robot to achieve manipulation-ready positioning using only RGB observations from its onboard cameras. Our method conditions the navigation policy on three inputs: goal images, multi-view RGB observations from the onboard cameras, and a text prompt specifying the target object. A language-driven segmentation module and a spatial score-matrix decoder then supply explicit object grounding and relative pose reasoning. Using real-world data from a single object instance within a category, the system generalizes to unseen object instances across diverse environments with challenging lighting and background conditions. To comprehensively evaluate this, we introduce two metrics: an edge-alignment metric, which uses ground truth orientation, and an object-alignment metric, which evaluates how well the robot visually faces the target. Under these metrics, our policy achieves 73.47% success in edge-alignment and 96.94% success in object-alignment when positioning relative to unseen target objects. These results show that precise last-meter navigation can be achieved at a category-level without depth, LiDAR, or map priors, enabling a scalable pathway toward unified mobile manipulation. Project page: https://rpm-lab-umn.github.io/category-level-last-meter-nav/

Related papers

• StereoPatch: Patch-Aligned RGB-Depth Fusion for Spatial Perception in Robot Manipulation [4.871965587776022]
  We introduce StereoPatch, a patch-aligned RGB-depth representation that binds registered metric geometry directly to the RGB patches used for action prediction.<n>On a shared 2-D patch grid, asymmetric cross-attention incorporates depth information into the corresponding RGB features before action decoding.<n>Across six real-robot tasks, StereoPatch achieves higher closed-loop success than appearance-only, geometry-only, raw RGB-D, and late-fusion baselines.
  arXiv  Detail & Related papers  (2026-09-14T13:01:25Z)
• LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation [52.91080323083409]
  Embodied navigation requires agents to translate heterogeneous goals and visual observations into actions across tasks, environments, and robot embodiments.<n>Here we present LightNav-0, a compact generalist embodied navigation model that elicits the spatial intelligence of a pretrained VLM and aligns it with navigation, without task-specific prediction heads.
  arXiv  Detail & Related papers  (2026-08-31T15:08:53Z)
• UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation [42.39025692205837]
  We propose UniLM-Nav, a framework for zero-shot open-vocabulary last-mile navigation.<n>UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and base-pose reasoning.<n>We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points.
  arXiv  Detail & Related papers  (2026-07-07T17:42:08Z)
• Revisiting Articulated Parts Perception in Robot Manipulation [53.043391395657835]
  Geometric Primary Structure (GPS) is an abstraction of the part geometry structure to balance scalability and quality.<n>GPS is integrated with a portable Virtual Reality (VR) device and requires only one minute to annotate one object sequence.<n>With this efficient VR-GPS system, we collect 41K frames for 234 objects across six part classes, and train a generalizable GPS model with a single RGB-D object image as input.
  arXiv  Detail & Related papers  (2026-06-06T11:07:29Z)
• ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigation [53.95797153529148]
  Embodied agents often struggle with efficient navigation because they rely primarily on partial egocentric observations.<n>We introduce ReasonNavi, a human-inspired framework that operationalizes this reason-then-act paradigm by coupling Multimodal Large Language Models (MLLMs) with deterministic planners.
  arXiv  Detail & Related papers  (2026-01-26T19:09:20Z)
• MetricNet: Recovering Metric Scale in Generative Navigation Policies [51.90872764552077]
  MetricNet is an effective add-on for generative navigation that predicts the metric distance between waypoints.<n>We show that executing MetricNet-scaled waypoints significantly improves both navigation and exploration performance.<n>We also propose MetricNav, which integrates MetricNet into a navigation policy to guide the robot away from obstacles while still moving towards the goal.
  arXiv  Detail & Related papers  (2025-09-17T13:37:13Z)
• TANGO: Traversability-Aware Navigation with Local Metric Control for Topological Goals [10.69725316052444]
  We present a novel RGB-only, object-level topometric navigation pipeline that enables zero-shot, long-horizon robot navigation.<n>Our approach integrates global topological path planning with local metric trajectory control, allowing the robot to navigate towards object-level sub-goals while avoiding obstacles.<n>We demonstrate the effectiveness of our method in both simulated environments and real-world tests, highlighting its robustness and deployability.
  arXiv  Detail & Related papers  (2025-09-10T15:43:32Z)
• Pointing-Guided Target Estimation via Transformer-Based Attention [8.35701920541908]
  Deictic gestures, like pointing, are a fundamental form of non-verbal communication, enabling humans to direct attention to specific objects or locations.<n>This capability is essential in Human-Robot Interaction (HRI), where robots should be able to predict human intent and anticipate appropriate responses.<n>We propose the Multi-Modality Inter-TransFormer (MM-ITF), a modular architecture to predict objects in a controlled tabletop scenario with the NICOL robot.
  arXiv  Detail & Related papers  (2025-09-05T11:42:03Z)
• MoMa-Kitchen: A 100K+ Benchmark for Affordance-Grounded Last-Mile Navigation in Mobile Manipulation [46.30498198691935]
  MoMa-Kitchen is a benchmark dataset for training models to learn optimal final navigation positions.<n>Visual data is collected from RGB-D inputs captured by a first-person view camera mounted on the robotic arm.<n>Our approach enables models to learn affordance-based final positioning that accommodates different arm types and platform heights.
  arXiv  Detail & Related papers  (2025-03-14T04:47:38Z)
• TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigation [52.422619828854984]
  We introduce TopV-Nav, an MLLM-based method that directly reasons on the top-view map with sufficient spatial information.<n>To fully unlock the MLLM's spatial reasoning potential in top-view perspective, we propose the Adaptive Visual Prompt Generation (AVPG) method.
  arXiv  Detail & Related papers  (2024-11-25T14:27:55Z)
• Interactive Semantic Map Representation for Skill-based Visual Object Navigation [43.71312386938849]
  This paper introduces a new representation of a scene semantic map formed during the embodied agent interaction with the indoor environment. We have implemented this representation into a full-fledged navigation approach called SkillTron. The proposed approach makes it possible to form both intermediate goals for robot exploration and the final goal for object navigation.
  arXiv  Detail & Related papers  (2023-11-07T16:30:12Z)
• NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration [57.15811390835294]
  This paper describes how we can train a single unified diffusion policy to handle both goal-directed navigation and goal-agnostic exploration. We show that this unified policy results in better overall performance when navigating to visually indicated goals in novel environments. Our experiments, conducted on a real-world mobile robot platform, show effective navigation in unseen environments in comparison with five alternative methods.
  arXiv  Detail & Related papers  (2023-10-11T21:07:14Z)
• Object-and-Action Aware Model for Visual Language Navigation [70.33142095637515]
  Vision-and-Language Navigation (VLN) is unique in that it requires turning relatively general natural-language instructions into robot agent actions. We propose an Object-and-Action Aware Model (OAAM) that processes these two different forms of natural language based instruction separately. This enables each process to match object-centered/action-centered instruction to their own counterpart visual perception/action orientation flexibly.
  arXiv  Detail & Related papers  (2020-07-29T06:32:18Z)
• Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework using Visual and Depth Cues [12.984393386954219]
  This paper addresses the problem of building augmented metric representations of scenes with semantic information from RGB-D images. We propose a complete framework to create an enhanced map representation of the environment with object-level information.
  arXiv  Detail & Related papers  (2020-03-13T15:05:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.