TDANet: Target-Directed Attention Network For Object-Goal Visual Navigation With Zero-Shot Ability

• URL: http://arxiv.org/abs/2404.08353v1
• Date: Fri, 12 Apr 2024 09:44:18 GMT
• Title: TDANet: Target-Directed Attention Network For Object-Goal Visual Navigation With Zero-Shot Ability
• Authors: Shiwei Lian, Feitian Zhang,
• Abstract summary: A target-directed attention network (TDANet) is proposed to learn the end-to-end object-goal visual navigation policy. TDANet features a novel target attention (TA) module that learns both the spatial and semantic relationships among objects to help TDANet focus on the most relevant observed objects to the target. To evaluate the navigation performance of TDANet, extensive experiments are conducted in the AI2-THOR embodied AI environment.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The generalization of the end-to-end deep reinforcement learning (DRL) for object-goal visual navigation is a long-standing challenge since object classes and placements vary in new test environments. Learning domain-independent visual representation is critical for enabling the trained DRL agent with the ability to generalize to unseen scenes and objects. In this letter, a target-directed attention network (TDANet) is proposed to learn the end-to-end object-goal visual navigation policy with zero-shot ability. TDANet features a novel target attention (TA) module that learns both the spatial and semantic relationships among objects to help TDANet focus on the most relevant observed objects to the target. With the Siamese architecture (SA) design, TDANet distinguishes the difference between the current and target states and generates the domain-independent visual representation. To evaluate the navigation performance of TDANet, extensive experiments are conducted in the AI2-THOR embodied AI environment. The simulation results demonstrate a strong generalization ability of TDANet to unseen scenes and target objects, with higher navigation success rate (SR) and success weighted by length (SPL) than other state-of-the-art models.

Related papers

• SemNav: A Model-Based Planner for Zero-Shot Object Goal Navigation Using Vision-Foundation Models [10.671262416557704]
  Vision Foundation Models (VFMs) offer powerful capabilities for visual understanding and reasoning.<n>We present a zero-shot object goal navigation framework that integrates the perceptual strength of VFMs with a model-based planner.<n>We evaluate our approach on the HM3D dataset using the Habitat simulator and demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-06-04T03:04:54Z)
• Object-Aware DINO (Oh-A-Dino): Enhancing Self-Supervised Representations for Multi-Object Instance Retrieval [1.4272411349249627]
  Self-supervised vision models like DINO have shown emergent object understanding. DINO representations excel at capturing global object attributes but struggle with object-level details like colour. We propose a method that combines global and local features by augmenting DINO representations with object-centric latent vectors.
  arXiv  Detail & Related papers  (2025-03-12T21:57:41Z)
• Learning Object-Centric Representation via Reverse Hierarchy Guidance [73.05170419085796]
  Object-Centric Learning (OCL) seeks to enable Neural Networks to identify individual objects in visual scenes. RHGNet introduces a top-down pathway that works in different ways in the training and inference processes. Our model achieves SOTA performance on several commonly used datasets.
  arXiv  Detail & Related papers  (2024-05-17T07:48:27Z)
• GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation [65.71524410114797]
  GOAT-Bench is a benchmark for the universal navigation task GO to AnyThing (GOAT) In GOAT, the agent is directed to navigate to a sequence of targets specified by the category name, language description, or image. We benchmark monolithic RL and modular methods on the GOAT task, analyzing their performance across modalities.
  arXiv  Detail & Related papers  (2024-04-09T20:40:00Z)
• VANP: Learning Where to See for Navigation with Self-Supervised Vision-Action Pre-Training [8.479135285935113]
  Humans excel at efficiently navigating through crowds without collision by focusing on specific visual regions relevant to navigation. Most robotic visual navigation methods rely on deep learning models pre-trained on vision tasks, which prioritize salient objects. We propose a Self-Supervised Vision-Action Model for Visual Navigation Pre-Training (VANP)
  arXiv  Detail & Related papers  (2024-03-12T22:33:08Z)
• Zero-Shot Object Goal Visual Navigation With Class-Independent Relationship Network [3.0820097046465285]
  "Zero-shot" means that the target the agent needs to find is not trained during the training phase. We propose the Class-Independent Relationship Network (CIRN) to address the issue of coupling navigation ability with target features during training. Our method outperforms the current state-of-the-art approaches in the zero-shot object goal visual navigation task.
  arXiv  Detail & Related papers  (2023-10-15T16:42:14Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigation [58.3480730643517]
  We present LGX, a novel algorithm for Language-Driven Zero-Shot Object Goal Navigation (L-ZSON) Our approach makes use of Large Language Models (LLMs) for this task. We achieve state-of-the-art zero-shot object navigation results on RoboTHOR with a success rate (SR) improvement of over 27% over the current baseline.
  arXiv  Detail & Related papers  (2023-03-06T20:19:19Z)
• Network Comparison Study of Deep Activation Feature Discriminability with Novel Objects [0.5076419064097732]
  State-of-the-art computer visions algorithms have incorporated Deep Neural Networks (DNN) in feature extracting roles, creating Deep Convolutional Activation Features (DeCAF) This study analyzes the general discriminability of novel object visual appearances encoded into the DeCAF space of six of the leading visual recognition DNN architectures.
  arXiv  Detail & Related papers  (2022-02-08T07:40:53Z)
• VTNet: Visual Transformer Network for Object Goal Navigation [36.15625223586484]
  We introduce a Visual Transformer Network (VTNet) for learning informative visual representation in navigation. In a nutshell, VTNet embeds object and region features with their location cues as spatial-aware descriptors. Experiments in the artificial environment AI2-Thor demonstrate that VTNet significantly outperforms state-of-the-art methods in unseen testing environments.
  arXiv  Detail & Related papers  (2021-05-20T01:23:15Z)
• Location-Sensitive Visual Recognition with Cross-IOU Loss [177.86369890708457]
  This paper proposes a unified solution named location-sensitive network (LSNet) for object detection, instance segmentation, and pose estimation. Based on a deep neural network as the backbone, LSNet predicts an anchor point and a set of landmarks which together define the shape of the target object.
  arXiv  Detail & Related papers  (2021-04-11T02:17:14Z)
• Improving Target-driven Visual Navigation with Attention on 3D Spatial Relationships [52.72020203771489]
  We investigate target-driven visual navigation using deep reinforcement learning (DRL) in 3D indoor scenes. Our proposed method combines visual features and 3D spatial representations to learn navigation policy. Our experiments, performed in the AI2-THOR, show that our model outperforms the baselines in both SR and SPL metrics.
  arXiv  Detail & Related papers  (2020-04-29T08:46:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.