G2-Nav: Grounded and Guarded Vision-Language Costmaps for Robot Social Navigation

• URL: http://arxiv.org/abs/2607.16956v1
• Date: Sat, 18 Jul 2026 20:18:20 GMT
• Title: G2-Nav: Grounded and Guarded Vision-Language Costmaps for Robot Social Navigation
• Abstract summary: Social navigation requires the robot to reason and respond in complex real-world environments.<n>We present G2-Nav, a novel framework that grounds abstract social reasoning and guards safe real-world deployment.<n>We demonstrate through real-world experiments that G2-Nav delivers safe, efficient, and socially compliant autonomous navigation in unstructured environments.
• Score: 28.188751747297715
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social navigation requires the robot to reason and respond in complex real-world environments. While recent works attempt to incorporate human-level intelligence into robot planning using large Vision-Language Models (VLMs), end-to-end frameworks often create an unpredictable black-box, and existing instruction-following methods are not designed for full autonomy. To bridge this gap, we present G2-Nav, a novel framework that grounds abstract social reasoning and guards safe real-world deployment. Instead of asking the VLM for direct planning decisions, G2-Nav translates its semantic reasoning into a vision-language costmap with reliability and interpretability. The VLM evaluates traversable regions and social agents from open-set perception, mapping social context into the costmap. To improve real-world robustness, the VLM performs semantic verification on upstream tracking, and we introduce a high-frequency safety check to guard against system latency prior to trajectory generation. We demonstrate through real-world experiments that G2-Nav delivers safe, efficient, and socially compliant autonomous navigation in unstructured environments. Code will be made publicly available.

Related papers

• From Language to Navigation Goals: A Vision-Language Approach for Semantic Navigation of Mobile Robots Using RGB-D Perception [42.903763223989536]
  This work presents a language-driven navigation framework that enables mobile robots to interpret user requests in natural language.<n>The framework is composed of modular ROS 2 components that cooperate to transform natural language instructions into navigation actions.<n>Results show that the framework successfully interprets both direct commands and contextual requests, generates meaningful natural-language feedback, and navigates towards the desired target.
  arXiv  Detail & Related papers  (2026-07-15T09:15:21Z)
• Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control [27.669648439283936]
  Social robot navigation (SRN) requires understanding human intentions, social norms, and contextual cues to generate socially compliant behaviors.<n>Recent advances in Vision-Language Models (VLMs) have opened new opportunities for SRN.<n>Survey aims to provide a foundation for building reliable, socially compliant, and deployable VLM-enabled navigation systems.
  arXiv  Detail & Related papers  (2026-06-27T06:32:47Z)
• Walk With Me: Long-Horizon Social Navigation for Human-Centric Outdoor Assistance [70.01362005983516]
  Walk with Me is a map-free framework for long-horizon social navigation from high-level human instructions.<n>It combines semantic intent grounding, map-free long-horizon planning, safety-aware reasoning, and low-level action generation.
  arXiv  Detail & Related papers  (2026-04-29T16:02:13Z)
• OpenFrontier: General Navigation with Visual-Language Grounded Frontiers [54.661157616245966]
  Open-world navigation requires robots to make decisions in complex everyday environments.<n>Recent advances in vision--language navigation (VLN) and vision--language--action (VLA) models enable end-to-end policies conditioned on natural language.<n>We propose OpenFrontier, a training-free navigation framework that seamlessly integrates diverse vision--language prior models.
  arXiv  Detail & Related papers  (2026-03-05T17:02:22Z)
• AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild [62.47761809929869]
  Vision-language navigation (VLN) requires intelligent agents to navigate environments by interpreting linguistic instructions alongside visual observations.<n>Current VLN research for unmanned aerial vehicles (UAVs) relies on detailed, pre-specified instructions to guide the UAV along predetermined routes.<n>We propose AutoFly, an end-to-end Vision-Language-Action model for autonomous UAV navigation.
  arXiv  Detail & Related papers  (2026-02-10T11:08:07Z)
• SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigation [32.75496547879437]
  Social navigation in dynamic, human-centered environments requires socially-compliant decisions grounded in robust scene understanding.<n>Recent Vision-Language Models (VLMs) exhibit promising capabilities that align with the nuanced requirements of social robot navigation.<n>We introduce the Social Navigation Scene Understanding Benchmark (SocialNav-SUB), a dataset and benchmark designed to evaluate VLMs for scene understanding.
  arXiv  Detail & Related papers  (2025-09-10T16:47:00Z)
• Hyp2Nav: Hyperbolic Planning and Curiosity for Crowd Navigation [58.574464340559466]
  We advocate for hyperbolic learning to enable crowd navigation and we introduce Hyp2Nav. Hyp2Nav leverages the intrinsic properties of hyperbolic geometry to better encode the hierarchical nature of decision-making processes in navigation tasks. We propose a hyperbolic policy model and a hyperbolic curiosity module that results in effective social navigation, best success rates, and returns across multiple simulation settings.
  arXiv  Detail & Related papers  (2024-07-18T14:40:33Z)
• Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
  Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
  arXiv  Detail & Related papers  (2024-07-02T21:00:30Z)
• Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Models [8.668211481067457]
  Co-NavGPT is a novel framework that integrates a Vision Language Model (VLM) as a global planner.<n>Co-NavGPT aggregates sub-maps from multiple robots with diverse viewpoints into a unified global map.<n>The VLM uses this information to assign frontiers across the robots, facilitating coordinated and efficient exploration.
  arXiv  Detail & Related papers  (2023-10-11T23:17:43Z)
• CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation [73.78984332354636]
  CorNav is a novel zero-shot framework for vision-and-language navigation. It incorporates environmental feedback for refining future plans and adjusting its actions. It consistently outperforms all baselines in a zero-shot multi-task setting.
  arXiv  Detail & Related papers  (2023-06-17T11:44:04Z)
• ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments [56.194988818341976]
  Vision-language navigation is a task that requires an agent to follow instructions to navigate in environments. We propose ETPNav, which focuses on two critical skills: 1) the capability to abstract environments and generate long-range navigation plans, and 2) the ability of obstacle-avoiding control in continuous environments. ETPNav yields more than 10% and 20% improvements over prior state-of-the-art on R2R-CE and RxR-CE datasets.
  arXiv  Detail & Related papers  (2023-04-06T13:07:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.