Two-Stage Depth Enhanced Learning with Obstacle Map For Object Navigation

• URL: http://arxiv.org/abs/2406.14103v1
• Date: Thu, 20 Jun 2024 08:35:10 GMT
• Title: Two-Stage Depth Enhanced Learning with Obstacle Map For Object Navigation
• Authors: Yanwei Zheng, Shaopu Feng, Bowen Huang, Changrui Li, Xiao Zhang, Dongxiao Yu,
• Abstract summary: This paper uses the RGB and depth information of the training scene to pretrain the feature extractor, which improves navigation efficiency. We evaluated our method on AI2-Thor and RoboTHOR and demonstrated that it significantly outperforms state-of-the-art (SOTA) methods on success rate and navigation efficiency.
• Score: 11.667940255053582
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The task that requires an agent to navigate to a given object through only visual observation is called visual object navigation (VON). The main bottlenecks of VON are strategies exploration and prior knowledge exploitation. Traditional strategies exploration ignores the differences of searching and navigating stages, using the same reward in two stages, which reduces navigation performance and training efficiency. Our study enables the agent to explore larger area in searching stage and seek the optimal path in navigating stage, improving the success rate of navigation. Traditional prior knowledge exploitation focused on learning and utilizing object association, which ignored the depth and obstacle information in the environment. This paper uses the RGB and depth information of the training scene to pretrain the feature extractor, which improves navigation efficiency. The obstacle information is memorized by the agent during the navigation, reducing the probability of collision and deadlock. Depth, obstacle and other prior knowledge are concatenated and input into the policy network, and navigation actions are output under the training of two-stage rewards. We evaluated our method on AI2-Thor and RoboTHOR and demonstrated that it significantly outperforms state-of-the-art (SOTA) methods on success rate and navigation efficiency.

Related papers

• TINA: Think, Interaction, and Action Framework for Zero-Shot Vision Language Navigation [11.591176410027224]
  This paper presents a Vision-Language Navigation (VLN) agent based on Large Language Models (LLMs) We propose the Thinking, Interacting, and Action framework to compensate for the shortcomings of LLMs in environmental perception. Our approach also outperformed some supervised learning-based methods, highlighting its efficacy in zero-shot navigation.
  arXiv  Detail & Related papers  (2024-03-13T05:22:39Z)
• Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidance [16.57243997206754]
  We propose a novel implicit obstacle map-driven indoor navigation framework for robust obstacle avoidance. A non-local target memory aggregation module is designed to leverage a non-local network to model the intrinsic relationship between the target semantic and the target orientation clues.
  arXiv  Detail & Related papers  (2023-08-24T15:10:28Z)
• KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigation [61.08389704326803]
  Vision-and-language navigation (VLN) is the task to enable an embodied agent to navigate to a remote location following the natural language instruction in real scenes. Most of the previous approaches utilize the entire features or object-centric features to represent navigable candidates. We propose a Knowledge Enhanced Reasoning Model (KERM) to leverage knowledge to improve agent navigation ability.
  arXiv  Detail & Related papers  (2023-03-28T08:00:46Z)
• Holistic Deep-Reinforcement-Learning-based Training of Autonomous Navigation Systems [4.409836695738518]
  Deep Reinforcement Learning emerged as a promising approach for autonomous navigation of ground vehicles. In this paper, we propose a holistic Deep Reinforcement Learning training approach involving all entities of the navigation stack.
  arXiv  Detail & Related papers  (2023-02-06T16:52:15Z)
• Explore before Moving: A Feasible Path Estimation and Memory Recalling Framework for Embodied Navigation [117.26891277593205]
  We focus on the navigation and solve the problem of existing navigation algorithms lacking experience and common sense. Inspired by the human ability to think twice before moving and conceive several feasible paths to seek a goal in unfamiliar scenes, we present a route planning method named Path Estimation and Memory Recalling framework. We show strong experimental results of PEMR on the EmbodiedQA navigation task.
  arXiv  Detail & Related papers  (2021-10-16T13:30:55Z)
• Augmented reality navigation system for visual prosthesis [67.09251544230744]
  We propose an augmented reality navigation system for visual prosthesis that incorporates a software of reactive navigation and path planning. It consists on four steps: locating the subject on a map, planning the subject trajectory, showing it to the subject and re-planning without obstacles. Results show how our augmented navigation system help navigation performance by reducing the time and distance to reach the goals, even significantly reducing the number of obstacles collisions.
  arXiv  Detail & Related papers  (2021-09-30T09:41:40Z)
• Deep Learning for Embodied Vision Navigation: A Survey [108.13766213265069]
  "Embodied visual navigation" problem requires an agent to navigate in a 3D environment mainly rely on its first-person observation. This paper attempts to establish an outline of the current works in the field of embodied visual navigation by providing a comprehensive literature survey.
  arXiv  Detail & Related papers  (2021-07-07T12:09:04Z)
• MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigation [4.127128889779478]
  This work focuses on performing better or comparable to the existing learning-based solutions for visual navigation for autonomous agents. We propose a method to encode vital scene semantics into a semantically informed, top-down egocentric map representation. We conduct experiments on 3-D reconstructed indoor PointGoal visual navigation and demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2021-03-21T12:01:23Z)
• Language-guided Navigation via Cross-Modal Grounding and Alternate Adversarial Learning [66.9937776799536]
  The emerging vision-and-language navigation (VLN) problem aims at learning to navigate an agent to the target location in unseen photo-realistic environments. The main challenges of VLN arise mainly from two aspects: first, the agent needs to attend to the meaningful paragraphs of the language instruction corresponding to the dynamically-varying visual environments. We propose a cross-modal grounding module to equip the agent with a better ability to track the correspondence between the textual and visual modalities.
  arXiv  Detail & Related papers  (2020-11-22T09:13:46Z)
• Active Visual Information Gathering for Vision-Language Navigation [115.40768457718325]
  Vision-language navigation (VLN) is the task of entailing an agent to carry out navigational instructions inside photo-realistic environments. One of the key challenges in VLN is how to conduct a robust navigation by mitigating the uncertainty caused by ambiguous instructions and insufficient observation of the environment. This work draws inspiration from human navigation behavior and endows an agent with an active information gathering ability for a more intelligent VLN policy.
  arXiv  Detail & Related papers  (2020-07-15T23:54:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.