Differentiable Spatial Planning using Transformers

• URL: http://arxiv.org/abs/2112.01010v1
• Date: Thu, 2 Dec 2021 06:48:16 GMT
• Title: Differentiable Spatial Planning using Transformers
• Authors: Devendra Singh Chaplot, Deepak Pathak, Jitendra Malik
• Abstract summary: We propose Spatial Planning Transformers (SPT), which given an obstacle map learns to generate actions by planning over long-range spatial dependencies. In the setting where the ground truth map is not known to the agent, we leverage pre-trained SPTs in an end-to-end framework. SPTs outperform prior state-of-the-art differentiable planners across all the setups for both manipulation and navigation tasks.
• Score: 87.90709874369192
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of spatial path planning. In contrast to the classical solutions which optimize a new plan from scratch and assume access to the full map with ground truth obstacle locations, we learn a planner from the data in a differentiable manner that allows us to leverage statistical regularities from past data. We propose Spatial Planning Transformers (SPT), which given an obstacle map learns to generate actions by planning over long-range spatial dependencies, unlike prior data-driven planners that propagate information locally via convolutional structure in an iterative manner. In the setting where the ground truth map is not known to the agent, we leverage pre-trained SPTs in an end-to-end framework that has the structure of mapper and planner built into it which allows seamless generalization to out-of-distribution maps and goals. SPTs outperform prior state-of-the-art differentiable planners across all the setups for both manipulation and navigation tasks, leading to an absolute improvement of 7-19%.

Related papers

• PAS-SLAM: A Visual SLAM System for Planar Ambiguous Scenes [41.47703182059505]
  We propose a visual SLAM system based on planar features designed for planar ambiguous scenes. We present an integrated data association strategy that combines plane parameters, semantic information, projection IoU, and non-parametric tests. Finally, we design a set of multi-constraint factor graphs for camera pose optimization.
  arXiv  Detail & Related papers  (2024-02-09T01:34:26Z)
• LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning [65.86754998249224]
  We develop a novel hybrid planner that leverages a conventional rule-based planner in conjunction with an LLM-based planner. Our approach navigates complex scenarios which existing planners struggle with, produces well-reasoned outputs while also remaining grounded through working alongside the rule-based approach.
  arXiv  Detail & Related papers  (2023-12-30T02:53:45Z)
• Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty [56.30846158280031]
  Task planning for embodied AI has been one of the most challenging problems. We propose a task-agnostic method named 'planning as in-painting' The proposed framework achieves promising performances in various embodied AI tasks.
  arXiv  Detail & Related papers  (2023-12-02T10:07:17Z)
• Embodied Task Planning with Large Language Models [86.63533340293361]
  We propose a TAsk Planing Agent (TaPA) in embodied tasks for grounded planning with physical scene constraint. During inference, we discover the objects in the scene by extending open-vocabulary object detectors to multi-view RGB images collected in different achievable locations. Experimental results show that the generated plan from our TaPA framework can achieve higher success rate than LLaVA and GPT-3.5 by a sizable margin.
  arXiv  Detail & Related papers  (2023-07-04T17:58:25Z)
• PlanT: Explainable Planning Transformers via Object-Level Representations [64.93938686101309]
  PlanT is a novel approach for planning in the context of self-driving. PlanT is based on imitation learning with a compact object-level input representation. Our results indicate that PlanT can focus on the most relevant object in the scene, even when this object is geometrically distant.
  arXiv  Detail & Related papers  (2022-10-25T17:59:46Z)
• [Re] Differentiable Spatial Planning using Transformers [0.6562256987706128]
  The problem of spatial path planning in a differentiable way is considered. They show that their proposed method of using Spatial Planning Transformers outperforms prior data-driven models. We verify these claims by reproducing their experiments and testing their method on new data.
  arXiv  Detail & Related papers  (2022-08-19T20:14:29Z)
• Adaptive Selection of Informative Path Planning Strategies via Reinforcement Learning [6.015556590955814]
  "Local planning" approaches adopt various spatial ranges within which next sampling locations are prioritized to investigate their effects on the prediction performance as well as incurred travel distance. Experiments on use cases of temperature monitoring robots demonstrate that the dynamic mixtures of planners can not only generate sophisticated, informative plans but also ensure significantly reduced distances at no cost of prediction reliability.
  arXiv  Detail & Related papers  (2021-08-14T21:32:33Z)
• Motion Planning Transformers: One Model to Plan Them All [15.82728888674882]
  We propose a transformer-based approach for efficiently solving the complex motion planning problems. Our approach first identifies regions on the map using transformers to provide attention to map areas likely to include the best path, and then applies local planners to generate the final collision-free path.
  arXiv  Detail & Related papers  (2021-06-05T04:29:16Z)
• UAV Path Planning using Global and Local Map Information with Deep Reinforcement Learning [16.720630804675213]
  This work presents a method for autonomous UAV path planning based on deep reinforcement learning (DRL) We compare coverage path planning ( CPP), where the UAV's goal is to survey an area of interest to data harvesting (DH), where the UAV collects data from distributed Internet of Things (IoT) sensor devices. By exploiting structured map information of the environment, we train double deep Q-networks (DDQNs) with identical architectures on both distinctly different mission scenarios.
  arXiv  Detail & Related papers  (2020-10-14T09:59:10Z)
• Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning [78.65083326918351]
  We consider alternatives to an implicit sequential planning assumption. We propose Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS) for approximating the optimal plan. We show that this algorithmic flexibility over planning order leads to improved results in navigation tasks in grid-worlds.
  arXiv  Detail & Related papers  (2020-04-23T18:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.