Beyond Waypoints: Dual-Heatmap Grounding for Cross-Embodiment Semantic Navigation

• URL: http://arxiv.org/abs/2605.19420v1
• Date: Tue, 19 May 2026 06:12:59 GMT
• Title: Beyond Waypoints: Dual-Heatmap Grounding for Cross-Embodiment Semantic Navigation
• Abstract summary: In this work, we focus on the practical setting of in-FOV semantic navigation.<n>We propose a unified Vision-Language framework that abandons single-point regression in favor of a Dual-Heatmap representation.<n>Our framework achieves state-of-the-art performance among comparable 8B baselines.
• Score: 13.081539889943906
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Grounding open-ended semantic instructions into physically executable local goals is a fundamental challenge in human-robot interaction. While existing navigation frameworks often regress deterministic waypoints, this rigid formulation collapses spatial uncertainty and frequently targets non-traversable object centers, leading to severe execution failures. In this work, we focus on the practical setting of in-FOV semantic navigation, where a robot receives concise, interleaved multimodal (text and image) prompts. To bridge the gap between abstract semantic intent and physical reachability, we propose a unified Vision-Language framework that abandons single-point regression in favor of a Dual-Heatmap representation. Our framework predicts a navigation affordance heatmap that captures continuous reachable regions, coupled with a facing heatmap for orientation constraints. These dense outputs inherently function as a differentiable semantic potential field, integrating seamlessly with downstream local planners. To support this paradigm, we build a fully automated, foundation-model-assisted synthetic data pipeline and establish a comprehensive simulation benchmark. Extensive experiments demonstrate that our framework achieves state-of-the-art performance among comparable 8B baselines. Crucially, a feature-fusion study and simulation studies across diverse robot embodiments (Jetbot, H1, Aliengo) reveal that explicit heatmap prediction drastically improves the Affordance Rate (AR). By placing targets reliably in executable free space, our framework effectively mitigates the brittleness of point regression, offering a transferable path toward safe cross-embodiment semantic navigation.

Related papers

• EgoNav: Bridging Learned Waypoints and Geometry-Aware Local Control for Robust Indoor Navigation [7.207055231363046]
  Image-goal navigation using lightweight topological maps is a practical paradigm for indoor robot deployment.<n>EgoNav is a hierarchical system that generates candidates from semantically segmented traversable regions.<n>EgoNav consistently outperforms contemporary baselines in both success rate and path efficiency.
  arXiv  Detail & Related papers  (2026-08-26T11:13:07Z)
• CrossTracer: Cross-Embodiment Navigation via VLA Model Reasoning and Trace Residuals Adapting [51.52109090141233]
  CrossTracer is a hierarchical framework for cross-embodiment navigation.<n>It forms a unified pixel-space interface between semantic reasoning and physical grounding.<n>We evaluate CrossTracer on the NaviTrace benchmark, which tests whether a model can generate embodiment-consistent navigation traces.
  arXiv  Detail & Related papers  (2026-08-07T01:24:53Z)
• AutoPath: Learning Transferable Goal-Conditioned Stochastic Path Prior for Safe Navigation Without Human Demonstrations [103.61864552260316]
  Real-time navigation in dynamic environments requires collision-free and dynamically feasible motion under limited perception.<n>We formulate navigation as learning a transferable goal-aligned path prior that models a reusable distribution over goal-aligned geometry-consistent local paths conditioned on local observations.<n>We develop a structured prior learning framework that parameterizes local paths using a geometry-aware polar action manifold and incorporates risk-sensitive utility shaping with multi-goal distributional rollouts for stable and safety-aware planning.
  arXiv  Detail & Related papers  (2026-07-13T16:00:41Z)
• Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation [63.11032720821731]
  textitSAGE is a framework that enables agents to learn within a physics-grounded semantic abstraction rather than a photorealistic simulation.<n>We demonstrate that textitSAGE significantly improves planner-assisted embodied navigation, achieving a 53.21% LLM-Match Success Rate on A-EQA.
  arXiv  Detail & Related papers  (2026-05-11T07:34:30Z)
• InsTraj: Instructing Diffusion Models with Travel Intentions to Generate Real-world Trajectories [58.29205608173041]
  InsTraj is a novel framework that instructs diffusion models to generate high-fidelity trajectories directly from natural language descriptions.<n>We show that InsTraj significantly outperforms state-of-the-art methods in generating trajectories that are realistic, diverse, and semantically faithful to the input instructions.
  arXiv  Detail & Related papers  (2026-04-05T12:56:55Z)
• IntentReact: Guiding Reactive Object-Centric Navigation via Topological Intent [38.14789750420476]
  We propose IntentReact, an intent-conditioned object-centric navigation framework.<n>We show improved navigation success and execution quality compared to prior object-centric navigation methods.
  arXiv  Detail & Related papers  (2026-03-26T12:32:11Z)
• Hierarchical Trajectory Planning of Floating-Base Multi-Link Robot for Maneuvering in Confined Environments [9.325443556436436]
  Floating-base multi-link robots can change their shape during flight, making them well-suited for applications in confined environments.<n>This work introduces a hierarchical trajectory planning framework that integrates global guidance with configuration-aware local optimization.<n>To the best of our knowledge, this is the first planning framework for floating-base multi-link robots that has been demonstrated on a real robot.
  arXiv  Detail & Related papers  (2026-02-25T22:49:54Z)
• AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild [62.47761809929869]
  Vision-language navigation (VLN) requires intelligent agents to navigate environments by interpreting linguistic instructions alongside visual observations.<n>Current VLN research for unmanned aerial vehicles (UAVs) relies on detailed, pre-specified instructions to guide the UAV along predetermined routes.<n>We propose AutoFly, an end-to-end Vision-Language-Action model for autonomous UAV navigation.
  arXiv  Detail & Related papers  (2026-02-10T11:08:07Z)
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• MetricNet: Recovering Metric Scale in Generative Navigation Policies [51.90872764552077]
  MetricNet is an effective add-on for generative navigation that predicts the metric distance between waypoints.<n>We show that executing MetricNet-scaled waypoints significantly improves both navigation and exploration performance.<n>We also propose MetricNav, which integrates MetricNet into a navigation policy to guide the robot away from obstacles while still moving towards the goal.
  arXiv  Detail & Related papers  (2025-09-17T13:37:13Z)
• GoViG: Goal-Conditioned Visual Navigation Instruction Generation [69.79110149746506]
  We introduce Goal-Conditioned Visual Navigation Instruction Generation (GoViG), a new task that aims to autonomously generate precise and contextually coherent navigation instructions.<n>GoViG exclusively leverages raw egocentric visual data, substantially improving its adaptability to unseen and unstructured environments.
  arXiv  Detail & Related papers  (2025-08-13T07:05:17Z)
• IN-Sight: Interactive Navigation through Sight [20.184155117341497]
  IN-Sight is a novel approach to self-supervised path planning. It calculates traversability scores and incorporates them into a semantic map. To precisely navigate around obstacles, IN-Sight employs a local planner.
  arXiv  Detail & Related papers  (2024-08-01T07:27:54Z)
• Interactive Semantic Map Representation for Skill-based Visual Object Navigation [43.71312386938849]
  This paper introduces a new representation of a scene semantic map formed during the embodied agent interaction with the indoor environment. We have implemented this representation into a full-fledged navigation approach called SkillTron. The proposed approach makes it possible to form both intermediate goals for robot exploration and the final goal for object navigation.
  arXiv  Detail & Related papers  (2023-11-07T16:30:12Z)
• How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers [94.46825166907831]
  We present a training-free solution to tackle the object goal navigation problem in Embodied AI. Our method builds a structured scene representation based on the classic visual simultaneous localization and mapping (V-SLAM) framework. Our method propagates semantics on the scene graphs based on language priors and scene statistics to introduce semantic knowledge to the geometric frontiers.
  arXiv  Detail & Related papers  (2023-05-26T13:38:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.