PALMER: Perception-Action Loop with Memory for Long-Horizon Planning

• URL: http://arxiv.org/abs/2212.04581v1
• Date: Thu, 8 Dec 2022 22:11:49 GMT
• Title: PALMER: Perception-Action Loop with Memory for Long-Horizon Planning
• Authors: Onur Beker, Mohammad Mohammadi, Amir Zamir
• Abstract summary: We introduce a general-purpose planning algorithm called PALMER. Palmer combines classical sampling-based planning algorithms with learning-based perceptual representations. This creates a tight feedback loop between representation learning, memory, reinforcement learning, and sampling-based planning.
• Score: 1.5469452301122177
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To achieve autonomy in a priori unknown real-world scenarios, agents should be able to: i) act from high-dimensional sensory observations (e.g., images), ii) learn from past experience to adapt and improve, and iii) be capable of long horizon planning. Classical planning algorithms (e.g. PRM, RRT) are proficient at handling long-horizon planning. Deep learning based methods in turn can provide the necessary representations to address the others, by modeling statistical contingencies between observations. In this direction, we introduce a general-purpose planning algorithm called PALMER that combines classical sampling-based planning algorithms with learning-based perceptual representations. For training these perceptual representations, we combine Q-learning with contrastive representation learning to create a latent space where the distance between the embeddings of two states captures how easily an optimal policy can traverse between them. For planning with these perceptual representations, we re-purpose classical sampling-based planning algorithms to retrieve previously observed trajectory segments from a replay buffer and restitch them into approximately optimal paths that connect any given pair of start and goal states. This creates a tight feedback loop between representation learning, memory, reinforcement learning, and sampling-based planning. The end result is an experiential framework for long-horizon planning that is significantly more robust and sample efficient compared to existing methods.

Related papers

• Provably Efficient Representation Learning with Tractable Planning in Low-Rank POMDP [81.00800920928621]
  We study representation learning in partially observable Markov Decision Processes (POMDPs) We first present an algorithm for decodable POMDPs that combines maximum likelihood estimation (MLE) and optimism in the face of uncertainty (OFU) We then show how to adapt this algorithm to also work in the broader class of $gamma$-observable POMDPs.
  arXiv  Detail & Related papers  (2023-06-21T16:04:03Z)
• ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning [132.20119288212376]
  We propose a spatial-temporal feature learning scheme towards a set of more representative features for perception, prediction and planning tasks simultaneously. To the best of our knowledge, we are the first to systematically investigate each part of an interpretable end-to-end vision-based autonomous driving system.
  arXiv  Detail & Related papers  (2022-07-15T16:57:43Z)
• Representation, learning, and planning algorithms for geometric task and motion planning [24.862289058632186]
  We present a framework for learning to guide geometric task and motion planning (GTAMP) GTAMP is a subclass of task and motion planning in which the goal is to move multiple objects to target regions among movable obstacles. A standard graph search algorithm is not directly applicable, because GTAMP problems involve hybrid search spaces and expensive action feasibility checks.
  arXiv  Detail & Related papers  (2022-03-09T09:47:01Z)
• Integrating Deep Reinforcement and Supervised Learning to Expedite Indoor Mapping [0.0]
  We show that combining the two methods can shorten the mapping time, compared to frontier-based motion planning, by up to 75%. One is the use of deep reinforcement learning to train the motion planner. The second is the inclusion of a pre-trained generative deep neural network, acting as a map predictor.
  arXiv  Detail & Related papers  (2021-09-17T12:07:07Z)
• Model-Based Reinforcement Learning via Latent-Space Collocation [110.04005442935828]
  We argue that it is easier to solve long-horizon tasks by planning sequences of states rather than just actions. We adapt the idea of collocation, which has shown good results on long-horizon tasks in optimal control literature, to the image-based setting by utilizing learned latent state space models.
  arXiv  Detail & Related papers  (2021-06-24T17:59:18Z)
• Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors [124.30562402952319]
  The ability to predict and plan into the future is fundamental for agents acting in the world. Current learning approaches for visual prediction and planning fail on long-horizon tasks. We propose a framework for visual prediction and planning that is able to overcome both of these limitations.
  arXiv  Detail & Related papers  (2020-06-23T17:58:56Z)
• Plan2Vec: Unsupervised Representation Learning by Latent Plans [106.37274654231659]
  We introduce plan2vec, an unsupervised representation learning approach that is inspired by reinforcement learning. Plan2vec constructs a weighted graph on an image dataset using near-neighbor distances, and then extrapolates this local metric to a global embedding by distilling path-integral over planned path. We demonstrate the effectiveness of plan2vec on one simulated and two challenging real-world image datasets.
  arXiv  Detail & Related papers  (2020-05-07T17:52:23Z)
• Hallucinative Topological Memory for Zero-Shot Visual Planning [86.20780756832502]
  In visual planning (VP), an agent learns to plan goal-directed behavior from observations of a dynamical system obtained offline. Most previous works on VP approached the problem by planning in a learned latent space, resulting in low-quality visual plans. Here, we propose a simple VP method that plans directly in image space and displays competitive performance.
  arXiv  Detail & Related papers  (2020-02-27T18:54:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.