Semantic Environment Atlas for Object-Goal Navigation

• URL: http://arxiv.org/abs/2410.09081v1
• Date: Sat, 5 Oct 2024 00:37:15 GMT
• Title: Semantic Environment Atlas for Object-Goal Navigation
• Abstract summary: We introduce the Semantic Environment Atlas (SEA), a novel mapping approach designed to enhance visual navigation capabilities of embodied agents. The SEA integrates multiple semantic maps from various environments, retaining a memory of place-object relationships. Our method achieves a success rate of 39.0%, an improvement of 12.4% over the current state-of-the-art, but also maintains robustness under noisy odometry and actuation conditions.
• Score: 12.057544558656035
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we introduce the Semantic Environment Atlas (SEA), a novel mapping approach designed to enhance visual navigation capabilities of embodied agents. The SEA utilizes semantic graph maps that intricately delineate the relationships between places and objects, thereby enriching the navigational context. These maps are constructed from image observations and capture visual landmarks as sparsely encoded nodes within the environment. The SEA integrates multiple semantic maps from various environments, retaining a memory of place-object relationships, which proves invaluable for tasks such as visual localization and navigation. We developed navigation frameworks that effectively leverage the SEA, and we evaluated these frameworks through visual localization and object-goal navigation tasks. Our SEA-based localization framework significantly outperforms existing methods, accurately identifying locations from single query images. Experimental results in Habitat scenarios show that our method not only achieves a success rate of 39.0%, an improvement of 12.4% over the current state-of-the-art, but also maintains robustness under noisy odometry and actuation conditions, all while keeping computational costs low.

Related papers

• IMPRINT: Image-Conditioned Query Enrichment for Long-Tail Object Goal Navigation [13.183204553352802]
  Embodied AI increasingly relies on queryable semantic maps built from pre-trained vision-language models to enable zero-shot Object Goal Navigation (ObjectNav)<n>We introduce IMPRINT, a zero-shot framework that enriches textual object queries with web-sourced images to improve grounding in queryable maps.<n>We present HSSD-rare, a new ObjectNav benchmark built on Habitat Synthetic Scenes featuring semantically specific subcategories.
  arXiv  Detail & Related papers  (2026-07-27T22:09:32Z)
• SR-Nav: Spatial Relationships Matter for Zero-shot Object Goal Navigation [23.52929533412924]
  Zero-shot object-goal navigation aims to find target objects in unseen environments using only egocentric observation.<n>Recent methods leverage foundation models' comprehension and reasoning capabilities to enhance navigation performance.<n>We propose Spatial Relation-aware Navigation (SR-Nav), a framework that models both observed and experience-based spatial relationships to enhance both perception and planning.
  arXiv  Detail & Related papers  (2026-03-19T03:09:32Z)
• SignNav: Leveraging Signage for Semantic Visual Navigation in Large-Scale Indoor Environments [57.79171900005793]
  Humans routinely leverage semantic hints provided by signage to navigate to destinations within Large-Scale Indoor environments.<n>This paper introduces a novel embodied navigation task, SignNav, which requires the agent to interpret semantic hint from signage and reason about the subsequent action based on current observation.<n>We propose the spatial-aware module grounds the semantic hint of signage into physical world, while the temporal-aware module captures long-range dependencies between historical states and current observation.
  arXiv  Detail & Related papers  (2026-03-17T06:36:26Z)
• Follow the Signs: Using Textual Cues and LLMs to Guide Efficient Robot Navigation [8.782169490089673]
  We propose a novel semantic navigation framework that infers patterns from partial observations and predicts regions where the goal is most likely located.<n>Our method combines local perceptual inputs with frontier-based exploration and periodic LLM queries.<n>We demonstrate that this approach enables more efficient navigation in sparse, partially observable grid environments by exploiting symbolic patterns.
  arXiv  Detail & Related papers  (2026-01-10T18:47:25Z)
• Nav-$R^2$ Dual-Relation Reasoning for Generalizable Open-Vocabulary Object-Goal Navigation [67.68165784193556]
  Nav-$R2$ is a framework that explicitly models two types of relationships, target-environment modeling and environment-action planning.<n>Our SA-Mem preserves the most target-relevant and current observation-relevant features from both temporal and semantic perspectives.<n>Nav-R2 achieves state-of-the-art performance in localizing unseen objects through a streamlined and efficient pipeline.
  arXiv  Detail & Related papers  (2025-12-02T04:21:02Z)
• Object Navigation with Structure-Semantic Reasoning-Based Multi-level Map and Multimodal Decision-Making LLM [18.406869393228813]
  We propose an active object navigation framework with Environmental Attributes Map (EAM) and MLLM Hierarchical Reasoning module (MHR)<n>EAM is constructed by reasoning observed environments with SBERT and predicting unobserved ones with Diffusion.<n>MHR is inspired by EAM to perform frontier exploration decision-making, avoiding the circuitous trajectories in long-range scenarios to improve path efficiency.
  arXiv  Detail & Related papers  (2025-06-06T09:08:40Z)
• MID: A Comprehensive Shore-Based Dataset for Multi-Scale Dense Ship Occlusion and Interaction Scenarios [10.748210940033484]
  The Maritime Ship Navigation Behavior dataset (MID) is designed to address challenges in ship detection within complex maritime environments. MID contains 5,673 images with 135,884 finely annotated target instances, supporting both supervised and semi-supervised learning. MID's images are sourced from high-definition video clips of real-world navigation across 43 water areas, with varied weather and lighting conditions.
  arXiv  Detail & Related papers  (2024-12-08T09:34:23Z)
• Real-Time Metric-Semantic Mapping for Autonomous Navigation in Outdoor Environments [18.7565126823704]
  We introduce an online metric-semantic mapping system that generates a global metric-semantic mesh map of large-scale outdoor environments. Our mapping process achieves exceptional speed, with frame processing taking less than 7ms, regardless of scenario scale. We integrate the resultant map into a real-world navigation system, enabling metric-semantic-based terrain assessment and autonomous point-to-point navigation within a campus environment.
  arXiv  Detail & Related papers  (2024-11-30T00:05:10Z)
• Diffusion as Reasoning: Enhancing Object Navigation via Diffusion Model Conditioned on LLM-based Object-Room Knowledge [9.465351278799016]
  We propose a novel approach to enhancing the ObjectNav task.<n>We train a diffusion model to learn the statistical distribution patterns of objects in semantic maps.<n>Using the map of the explored regions during navigation as the condition to generate the map of the unknown regions, we realize the long-term goal reasoning of the target object.
  arXiv  Detail & Related papers  (2024-10-29T08:10:06Z)
• Mapping High-level Semantic Regions in Indoor Environments without Object Recognition [50.624970503498226]
  The present work proposes a method for semantic region mapping via embodied navigation in indoor environments. To enable region identification, the method uses a vision-to-language model to provide scene information for mapping. By projecting egocentric scene understanding into the global frame, the proposed method generates a semantic map as a distribution over possible region labels at each location.
  arXiv  Detail & Related papers  (2024-03-11T18:09:50Z)
• Learning Navigational Visual Representations with Semantic Map Supervision [85.91625020847358]
  We propose a navigational-specific visual representation learning method by contrasting the agent's egocentric views and semantic maps. Ego$2$-Map learning transfers the compact and rich information from a map, such as objects, structure and transition, to the agent's egocentric representations for navigation.
  arXiv  Detail & Related papers  (2023-07-23T14:01:05Z)
• Navigating to Objects in Unseen Environments by Distance Prediction [16.023495311387478]
  We propose an object goal navigation framework, which could directly perform path planning based on an estimated distance map. Specifically, our model takes a birds-eye-view semantic map as input, and estimates the distance from the map cells to the target object. With the estimated distance map, the agent could explore the environment and navigate to the target objects based on either human-designed or learned navigation policy.
  arXiv  Detail & Related papers  (2022-02-08T09:22:50Z)
• Learning to Map for Active Semantic Goal Navigation [40.193928212509356]
  We propose a novel framework that actively learns to generate semantic maps outside the field of view of the agent. We show how different objectives can be defined by balancing exploration with exploitation. Our method is validated in the visually realistic environments offered by the Matterport3D dataset.
  arXiv  Detail & Related papers  (2021-06-29T18:01:30Z)
• Structured Scene Memory for Vision-Language Navigation [155.63025602722712]
  We propose a crucial architecture for vision-language navigation (VLN) It is compartmentalized enough to accurately memorize the percepts during navigation. It also serves as a structured scene representation, which captures and disentangles visual and geometric cues in the environment.
  arXiv  Detail & Related papers  (2021-03-05T03:41:00Z)
• SVAM: Saliency-guided Visual Attention Modeling by Autonomous Underwater Robots [16.242924916178282]
  This paper presents a holistic approach to saliency-guided visual attention modeling (SVAM) for use by autonomous underwater robots. Our proposed model, named SVAM-Net, integrates deep visual features at various scales and semantics for effective salient object detection (SOD) in natural underwater images.
  arXiv  Detail & Related papers  (2020-11-12T08:17:21Z)
• Occupancy Anticipation for Efficient Exploration and Navigation [97.17517060585875]
  We propose occupancy anticipation, where the agent uses its egocentric RGB-D observations to infer the occupancy state beyond the visible regions. By exploiting context in both the egocentric views and top-down maps our model successfully anticipates a broader map of the environment. Our approach is the winning entry in the 2020 Habitat PointNav Challenge.
  arXiv  Detail & Related papers  (2020-08-21T03:16:51Z)
• Object Goal Navigation using Goal-Oriented Semantic Exploration [98.14078233526476]
  This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. We propose a modular system called, Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently.
  arXiv  Detail & Related papers  (2020-07-01T17:52:32Z)
• Neural Topological SLAM for Visual Navigation [112.73876869904]
  We design topological representations for space that leverage semantics and afford approximate geometric reasoning. We describe supervised learning-based algorithms that can build, maintain and use such representations under noisy actuation.
  arXiv  Detail & Related papers  (2020-05-25T17:56:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.