Meta-operators for Enabling Parallel Planning Using Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2403.08910v1
• Date: Wed, 13 Mar 2024 19:00:36 GMT
• Title: Meta-operators for Enabling Parallel Planning Using Deep Reinforcement Learning
• Authors: Ángel Aso-Mollar, Eva Onaindia,
• Abstract summary: We introduce the concept of meta-operator as the result of simultaneously applying multiple planning operators. We show that including meta-operators in the RL action space enables new planning perspectives to be addressed using RL, such as parallel planning.
• Score: 0.8287206589886881
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is a growing interest in the application of Reinforcement Learning (RL) techniques to AI planning with the aim to come up with general policies. Typically, the mapping of the transition model of AI planning to the state transition system of a Markov Decision Process is established by assuming a one-to-one correspondence of the respective action spaces. In this paper, we introduce the concept of meta-operator as the result of simultaneously applying multiple planning operators, and we show that including meta-operators in the RL action space enables new planning perspectives to be addressed using RL, such as parallel planning. Our research aims to analyze the performance and complexity of including meta-operators in the RL process, concretely in domains where satisfactory outcomes have not been previously achieved using usual generalized planning models. The main objective of this article is thus to pave the way towards a redefinition of the RL action space in a manner that is more closely aligned with the planning perspective.

Related papers

• Parallel Strategies for Best-First Generalized Planning [51.713634067802104]
  Generalized planning (GP) is a research area of AI that studies the automated synthesis of algorithmic-like solutions capable of solving multiple classical planning instances. One of the current advancements has been the introduction of Best-First Generalized Planning (BFGP), a GP algorithm based on a novel solution space that can be explored with search. This paper evaluates the application of parallel search techniques to BFGP, another critical component in closing the performance gap.
  arXiv  Detail & Related papers  (2024-07-31T09:50:22Z)
• From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems [59.40480894948944]
  Large language model (LLM) empowered agents are able to solve decision-making problems in the physical world. Under this model, the LLM Planner navigates a partially observable Markov decision process (POMDP) by iteratively generating language-based subgoals via prompting. We prove that the pretrained LLM Planner effectively performs Bayesian aggregated imitation learning (BAIL) through in-context learning.
  arXiv  Detail & Related papers  (2024-05-30T09:42:54Z)
• On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS) [23.024862968785147]
  This paper investigates eight categories based on the unique applications of LLMs in addressing various aspects of planning problems. A critical insight resulting from our review is that the true potential of LLMs unfolds when they are integrated with traditional symbolic planners.
  arXiv  Detail & Related papers  (2024-01-04T19:22:09Z)
• LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning [65.86754998249224]
  We develop a novel hybrid planner that leverages a conventional rule-based planner in conjunction with an LLM-based planner. Our approach navigates complex scenarios which existing planners struggle with, produces well-reasoned outputs while also remaining grounded through working alongside the rule-based approach.
  arXiv  Detail & Related papers  (2023-12-30T02:53:45Z)
• GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models [31.628341050846768]
  Goal-conditioned Offline Planning (GOPlan) is a novel model-based framework that contains two key phases. GOPlan pretrains a prior policy capable of capturing multi-modal action distribution within the multi-goal dataset. The reanalysis method generates high-quality imaginary data by planning with learned models for both intra-trajectory and inter-trajectory goals.
  arXiv  Detail & Related papers  (2023-10-30T21:19:52Z)
• Imitating Graph-Based Planning with Goal-Conditioned Policies [72.61631088613048]
  We present a self-imitation scheme which distills a subgoal-conditioned policy into the target-goal-conditioned policy. We empirically show that our method can significantly boost the sample-efficiency of the existing goal-conditioned RL methods.
  arXiv  Detail & Related papers  (2023-03-20T14:51:10Z)
• Differentiable Spatial Planning using Transformers [87.90709874369192]
  We propose Spatial Planning Transformers (SPT), which given an obstacle map learns to generate actions by planning over long-range spatial dependencies. In the setting where the ground truth map is not known to the agent, we leverage pre-trained SPTs in an end-to-end framework. SPTs outperform prior state-of-the-art differentiable planners across all the setups for both manipulation and navigation tasks.
  arXiv  Detail & Related papers  (2021-12-02T06:48:16Z)
• Deliberative Acting, Online Planning and Learning with Hierarchical Operational Models [5.597986898418404]
  In AI research, a plan of action has typically used descriptive models of the actions that abstractly specify what might happen as a result of an action. executing the planned actions has needed operational models, in which rich computational control structures and closed-loop online decision-making are used. We implement an integrated acting and planning system in which both planning and acting use the same operational models.
  arXiv  Detail & Related papers  (2020-10-02T14:50:05Z)
• Model-based Reinforcement Learning: A Survey [2.564530030795554]
  Sequential decision making, commonly formalized as Markov Decision Process (MDP) optimization, is a important challenge in artificial intelligence. Two key approaches to this problem are reinforcement learning (RL) and planning. This paper presents a survey of the integration of both fields, better known as model-based reinforcement learning.
  arXiv  Detail & Related papers  (2020-06-30T12:10:07Z)
• Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning [78.65083326918351]
  We consider alternatives to an implicit sequential planning assumption. We propose Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS) for approximating the optimal plan. We show that this algorithmic flexibility over planning order leads to improved results in navigation tasks in grid-worlds.
  arXiv  Detail & Related papers  (2020-04-23T18:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.