Differentiable Mapping Networks: Learning Structured Map Representations for Sparse Visual Localization

• URL: http://arxiv.org/abs/2005.09530v1
• Date: Tue, 19 May 2020 15:43:39 GMT
• Title: Differentiable Mapping Networks: Learning Structured Map Representations for Sparse Visual Localization
• Authors: Peter Karkus, Anelia Angelova, Vincent Vanhoucke, Rico Jonschkowski
• Abstract summary: Differentiable Mapping Network (DMN) learns effective map representations for visual localization. We evaluate the DMN using simulated environments and a challenging real-world Street View dataset.
• Score: 28.696160266177806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mapping and localization, preferably from a small number of observations, are fundamental tasks in robotics. We address these tasks by combining spatial structure (differentiable mapping) and end-to-end learning in a novel neural network architecture: the Differentiable Mapping Network (DMN). The DMN constructs a spatially structured view-embedding map and uses it for subsequent visual localization with a particle filter. Since the DMN architecture is end-to-end differentiable, we can jointly learn the map representation and localization using gradient descent. We apply the DMN to sparse visual localization, where a robot needs to localize in a new environment with respect to a small number of images from known viewpoints. We evaluate the DMN using simulated environments and a challenging real-world Street View dataset. We find that the DMN learns effective map representations for visual localization. The benefit of spatial structure increases with larger environments, more viewpoints for mapping, and when training data is scarce. Project website: http://sites.google.com/view/differentiable-mapping

Related papers

• Deep Homography Estimation for Visual Place Recognition [49.235432979736395]
  We propose a transformer-based deep homography estimation (DHE) network. It takes the dense feature map extracted by a backbone network as input and fits homography for fast and learnable geometric verification. Experiments on benchmark datasets show that our method can outperform several state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-25T13:22:17Z)
• Neural Implicit Dense Semantic SLAM [83.04331351572277]
  We propose a novel RGBD vSLAM algorithm that learns a memory-efficient, dense 3D geometry, and semantic segmentation of an indoor scene in an online manner. Our pipeline combines classical 3D vision-based tracking and loop closing with neural fields-based mapping. Our proposed algorithm can greatly enhance scene perception and assist with a range of robot control problems.
  arXiv  Detail & Related papers  (2023-04-27T23:03:52Z)
• highway2vec -- representing OpenStreetMap microregions with respect to their road network characteristics [3.5960954499553512]
  We propose a method for generating microregions' embeddings with respect to road infrastructure characteristics. We base our representations on OpenStreetMap road networks in a selection of cities. We obtained vector representations that detect how similar map hexagons are in the road networks they contain.
  arXiv  Detail & Related papers  (2023-04-26T23:16:18Z)
• Self-Supervised Feature Learning for Long-Term Metric Visual Localization [16.987148593917905]
  We present a novel self-supervised feature learning framework for metric visual localization. We use a sequence-based image matching algorithm to generate image correspondences without ground-truth labels. We can then sample image pairs to train a deep neural network that learns sparse features with associated descriptors and scores without ground-truth pose supervision.
  arXiv  Detail & Related papers  (2022-11-30T21:15:05Z)
• iSDF: Real-Time Neural Signed Distance Fields for Robot Perception [64.80458128766254]
  iSDF is a continuous learning system for real-time signed distance field reconstruction. It produces more accurate reconstructions and better approximations of collision costs and gradients.
  arXiv  Detail & Related papers  (2022-04-05T15:48:39Z)
• Hippocampal Spatial Mapping As Fast Graph Learning [0.0]
  The hippocampal formation is thought to learn spatial maps of environments, and in many models this learning process consists of forming a sensory association for each location in the environment. In this work, I approach spatial mapping as a problem of learning graphs of environment parts. Each node in the learned graph, represented by hippocampal engram cells, is associated with feature information in lateral entorhinal cortex (LEC) and location information in medial entorhinal cortex (MEC) using empirically observed neuron types. This core idea of associating arbitrary information with nodes and edges is not inherently spatial, so this proposed fast-
  arXiv  Detail & Related papers  (2021-07-01T16:05:42Z)
• Region Similarity Representation Learning [94.88055458257081]
  Region Similarity Representation Learning (ReSim) is a new approach to self-supervised representation learning for localization-based tasks. ReSim learns both regional representations for localization as well as semantic image-level representations. We show how ReSim learns representations which significantly improve the localization and classification performance compared to a competitive MoCo-v2 baseline.
  arXiv  Detail & Related papers  (2021-03-24T00:42:37Z)
• Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces [68.12457459590921]
  Reconstructing continuous surfaces from 3D point clouds is a fundamental operation in 3D geometry processing. We introduce textitNeural-Pull, a new approach that is simple and leads to high quality SDFs.
  arXiv  Detail & Related papers  (2020-11-26T23:18:10Z)
• Unsupervised Metric Relocalization Using Transform Consistency Loss [66.19479868638925]
  Training networks to perform metric relocalization traditionally requires accurate image correspondences. We propose a self-supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardless of the reference image used for registration. We evaluate our framework on synthetic and real-world data, showing our approach outperforms other supervised methods when a limited amount of ground-truth information is available.
  arXiv  Detail & Related papers  (2020-11-01T19:24:27Z)
• Supervised Topological Maps [0.76146285961466]
  Controlling the internal representation space of a neural network is a desirable feature because it allows to generate new data in a supervised manner. We will show how this can be achieved while building a low-dimensional mapping of the input stream, by deriving a generalized algorithm starting from Self Organizing Maps (SOMs)
  arXiv  Detail & Related papers  (2020-08-14T14:30:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.