Sparse Graphical Memory for Robust Planning

• URL: http://arxiv.org/abs/2003.06417v3
• Date: Thu, 12 Nov 2020 21:37:49 GMT
• Title: Sparse Graphical Memory for Robust Planning
• Authors: Scott Emmons, Ajay Jain, Michael Laskin, Thanard Kurutach, Pieter Abbeel, Deepak Pathak
• Abstract summary: We introduce Sparse Graphical Memory (SGM), a new data structure that stores states and feasible transitions in a sparse memory. SGM aggregates states according to a novel two-way consistency objective, adapting classic state aggregation criteria to goal-conditioned RL. We show that SGM significantly outperforms current state of the art methods on long horizon, sparse-reward visual navigation tasks.
• Score: 93.39298821537197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To operate effectively in the real world, agents should be able to act from high-dimensional raw sensory input such as images and achieve diverse goals across long time-horizons. Current deep reinforcement and imitation learning methods can learn directly from high-dimensional inputs but do not scale well to long-horizon tasks. In contrast, classical graphical methods like A* search are able to solve long-horizon tasks, but assume that the state space is abstracted away from raw sensory input. Recent works have attempted to combine the strengths of deep learning and classical planning; however, dominant methods in this domain are still quite brittle and scale poorly with the size of the environment. We introduce Sparse Graphical Memory (SGM), a new data structure that stores states and feasible transitions in a sparse memory. SGM aggregates states according to a novel two-way consistency objective, adapting classic state aggregation criteria to goal-conditioned RL: two states are redundant when they are interchangeable both as goals and as starting states. Theoretically, we prove that merging nodes according to two-way consistency leads to an increase in shortest path lengths that scales only linearly with the merging threshold. Experimentally, we show that SGM significantly outperforms current state of the art methods on long horizon, sparse-reward visual navigation tasks. Project video and code are available at https://mishalaskin.github.io/sgm/

Related papers

• Towards Synchronous Memorizability and Generalizability with Site-Modulated Diffusion Replay for Cross-Site Continual Segmentation [50.70671908078593]
  This paper proposes a novel training paradigm, learning towards Synchronous Memorizability and Generalizability (SMG-Learning) We create the orientational gradient alignment to ensure memorizability on previous sites, and arbitrary gradient alignment to enhance generalizability on unseen sites. Experimental results show that our method efficiently enhances both memorizability and generalizablity better than other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-26T03:10:57Z)
• GLACE: Global Local Accelerated Coordinate Encoding [66.87005863868181]
  Scene coordinate regression methods are effective in small-scale scenes but face significant challenges in large-scale scenes. We propose GLACE, which integrates pre-trained global and local encodings and enables SCR to scale to large scenes with only a single small-sized network. Our method achieves state-of-the-art results on large-scale scenes with a low-map-size model.
  arXiv  Detail & Related papers  (2024-06-06T17:59:50Z)
• Deep Homography Estimation for Visual Place Recognition [49.235432979736395]
  We propose a transformer-based deep homography estimation (DHE) network. It takes the dense feature map extracted by a backbone network as input and fits homography for fast and learnable geometric verification. Experiments on benchmark datasets show that our method can outperform several state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-25T13:22:17Z)
• Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings [23.1120983784623]
  quantization is a favored solution as it maps high precision tensors to a low precision, memory efficient format. In terms of memory footprint reduction, its most effective variants are based on codebooks. We propose a joint learning of the codebook and weight mappings that bears similarities with recent gradient-based post-training quantization techniques.
  arXiv  Detail & Related papers  (2023-09-29T16:04:55Z)
• One-4-All: Neural Potential Fields for Embodied Navigation [10.452316044889177]
  Real-world navigation can require long-horizon planning using high-dimensional RGB images. One-4-All (O4A) is a method leveraging self-supervised and manifold learning to obtain a graph-free, end-to-end navigation pipeline. We show that O4A can reach long-range goals in 8 simulated Gibson indoor environments.
  arXiv  Detail & Related papers  (2023-03-07T16:25:41Z)
• APP: Anytime Progressive Pruning [104.36308667437397]
  We propose a novel way of training a neural network with a target sparsity in a particular case of online learning: the anytime learning at macroscale paradigm (ALMA) The proposed approach significantly outperforms the baseline dense and Anytime OSP models across multiple architectures and datasets under short, moderate, and long-sequence training.
  arXiv  Detail & Related papers  (2022-04-04T16:38:55Z)
• Jointly Learnable Data Augmentations for Self-Supervised GNNs [0.311537581064266]
  We propose GraphSurgeon, a novel self-supervised learning method for graph representation learning. We take advantage of the flexibility of the learnable data augmentation and introduce a new strategy that augments in the embedding space. Our finding shows that GraphSurgeon is comparable to six SOTA semi-supervised and on par with five SOTA self-supervised baselines in node classification tasks.
  arXiv  Detail & Related papers  (2021-08-23T21:33:12Z)
• Model-Based Reinforcement Learning via Latent-Space Collocation [110.04005442935828]
  We argue that it is easier to solve long-horizon tasks by planning sequences of states rather than just actions. We adapt the idea of collocation, which has shown good results on long-horizon tasks in optimal control literature, to the image-based setting by utilizing learned latent state space models.
  arXiv  Detail & Related papers  (2021-06-24T17:59:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.