Renderable Neural Radiance Map for Visual Navigation

• URL: http://arxiv.org/abs/2303.00304v4
• Date: Thu, 20 Apr 2023 01:50:55 GMT
• Title: Renderable Neural Radiance Map for Visual Navigation
• Authors: Obin Kwon, Jeongho Park, Songhwai Oh
• Abstract summary: We propose a novel type of map for visual navigation, a renderable neural radiance map (RNR-Map) The RNR-Map has a grid form and consists of latent codes at each pixel. The recorded latent codes implicitly contain visual information about the environment, which makes the RNR-Map visually descriptive.
• Score: 18.903118231531973
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We propose a novel type of map for visual navigation, a renderable neural radiance map (RNR-Map), which is designed to contain the overall visual information of a 3D environment. The RNR-Map has a grid form and consists of latent codes at each pixel. These latent codes are embedded from image observations, and can be converted to the neural radiance field which enables image rendering given a camera pose. The recorded latent codes implicitly contain visual information about the environment, which makes the RNR-Map visually descriptive. This visual information in RNR-Map can be a useful guideline for visual localization and navigation. We develop localization and navigation frameworks that can effectively utilize the RNR-Map. We evaluate the proposed frameworks on camera tracking, visual localization, and image-goal navigation. Experimental results show that the RNR-Map-based localization framework can find the target location based on a single query image with fast speed and competitive accuracy compared to other baselines. Also, this localization framework is robust to environmental changes, and even finds the most visually similar places when a query image from a different environment is given. The proposed navigation framework outperforms the existing image-goal navigation methods in difficult scenarios, under odometry and actuation noises. The navigation framework shows 65.7% success rate in curved scenarios of the NRNS dataset, which is an improvement of 18.6% over the current state-of-the-art. Project page: https://rllab-snu.github.io/projects/RNR-Map/

Related papers

• Semantic Environment Atlas for Object-Goal Navigation [12.057544558656035]
  We introduce the Semantic Environment Atlas (SEA), a novel mapping approach designed to enhance visual navigation capabilities of embodied agents. The SEA integrates multiple semantic maps from various environments, retaining a memory of place-object relationships. Our method achieves a success rate of 39.0%, an improvement of 12.4% over the current state-of-the-art, but also maintains robustness under noisy odometry and actuation conditions.
  arXiv  Detail & Related papers  (2024-10-05T00:37:15Z)
• Visual Localization in 3D Maps: Comparing Point Cloud, Mesh, and NeRF Representations [8.522160106746478]
  We present a global visual localization system capable of localizing a single camera image across various 3D map representations. Our system generates a database by synthesizing novel views of the scene, creating RGB and depth image pairs. NeRF synthesized images show superior performance, localizing query images at an average success rate of 72%.
  arXiv  Detail & Related papers  (2024-08-21T19:37:17Z)
• MapLocNet: Coarse-to-Fine Feature Registration for Visual Re-Localization in Navigation Maps [8.373285397029884]
  Traditional localization approaches rely on high-definition (HD) maps, which consist of precisely annotated landmarks. We propose a novel transformer-based neural re-localization method, inspired by image registration. Our method significantly outperforms the current state-of-the-art OrienterNet on both the nuScenes and Argoverse datasets.
  arXiv  Detail & Related papers  (2024-07-11T14:51:18Z)
• GaussNav: Gaussian Splatting for Visual Navigation [92.13664084464514]
  Instance ImageGoal Navigation (IIN) requires an agent to locate a specific object depicted in a goal image within an unexplored environment. Our framework constructs a novel map representation based on 3D Gaussian Splatting (3DGS) Our framework demonstrates a significant leap in performance, evidenced by an increase in Success weighted by Path Length (SPL) from 0.252 to 0.578 on the challenging Habitat-Matterport 3D (HM3D) dataset.
  arXiv  Detail & Related papers  (2024-03-18T09:56:48Z)
• Language-enhanced RNR-Map: Querying Renderable Neural Radiance Field maps with natural language [51.805056586678184]
  We present a Language-enhanced Renderable Neural Radiance map for Visual Navigation with natural language query prompts. Le-RNR-Map employs a grid structure comprising latent codes positioned at each pixel. We enhance RNR-Map with CLIP-based embedding latent codes, allowing natural language search without additional label data.
  arXiv  Detail & Related papers  (2023-08-17T08:27:01Z)
• Neural Implicit Dense Semantic SLAM [83.04331351572277]
  We propose a novel RGBD vSLAM algorithm that learns a memory-efficient, dense 3D geometry, and semantic segmentation of an indoor scene in an online manner. Our pipeline combines classical 3D vision-based tracking and loop closing with neural fields-based mapping. Our proposed algorithm can greatly enhance scene perception and assist with a range of robot control problems.
  arXiv  Detail & Related papers  (2023-04-27T23:03:52Z)
• HPointLoc: Point-based Indoor Place Recognition using Synthetic RGB-D Images [58.720142291102135]
  We present a novel dataset named as HPointLoc, specially designed for exploring capabilities of visual place recognition in indoor environment. The dataset is based on the popular Habitat simulator, in which it is possible to generate indoor scenes using both own sensor data and open datasets.
  arXiv  Detail & Related papers  (2022-12-30T12:20:56Z)
• CroCo: Cross-Modal Contrastive learning for localization of Earth Observation data [62.96337162094726]
  It is of interest to localize a ground-based LiDAR point cloud on remote sensing imagery. We propose a contrastive learning-based method that trains on DEM and high-resolution optical imagery. In the best scenario, the Top-1 score of 0.71 and Top-5 score of 0.81 are obtained.
  arXiv  Detail & Related papers  (2022-04-14T15:55:00Z)
• Memory-Augmented Reinforcement Learning for Image-Goal Navigation [67.3963444878746]
  We present a novel method that leverages a cross-episode memory to learn to navigate. In order to avoid overfitting, we propose to use data augmentation on the RGB input during training. We obtain this competitive performance from RGB input only, without access to additional sensors such as position or depth.
  arXiv  Detail & Related papers  (2021-01-13T16:30:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.