GenPlan: Generative Sequence Models as Adaptive Planners

• URL: http://arxiv.org/abs/2412.08565v2
• Date: Wed, 25 Dec 2024 19:45:43 GMT
• Title: GenPlan: Generative Sequence Models as Adaptive Planners
• Authors: Akash Karthikeyan, Yash Vardhan Pant,
• Abstract summary: Sequence models have demonstrated remarkable success in behavioral planning by leveraging previously collected demonstrations. However, solving multi-task missions remains a significant challenge, particularly when the planner must adapt to unseen constraints and tasks. We propose GenPlan: a discrete-flow model for adaptive planner, enabling sample-generative exploration and exploitation.
• Score: 0.0
• License:
• Abstract: Sequence models have demonstrated remarkable success in behavioral planning by leveraging previously collected demonstrations. However, solving multi-task missions remains a significant challenge, particularly when the planner must adapt to unseen constraints and tasks, such as discovering goals and unlocking doors. Such behavioral planning problems are challenging to solve due to: a) agents failing to adapt beyond the single task learned through their reward function, and b) inability to generalize to new environments, e.g., those with walls and locked doors, when trained only in planar environments. Consequently, state-of-the-art decision-making methods are limited to missions where the required tasks are well-represented in the training demonstrations and can be solved within a short (temporal) planning horizon. To address this, we propose GenPlan: a stochastic and adaptive planner that leverages discrete-flow models for generative sequence modeling, enabling sample-efficient exploration and exploitation. This framework relies on an iterative denoising procedure to generate a sequence of goals and actions. This approach captures multi-modal action distributions and facilitates goal and task discovery, thereby generalizing to out-of-distribution tasks and environments, i.e., missions not part of the training data. We demonstrate the effectiveness of our method through multiple simulation environments. Notably, GenPlan outperforms state-of-the-art methods by over 10% on adaptive planning tasks, where the agent adapts to multi-task missions while leveraging demonstrations from single-goal-reaching tasks. Our code is available at https://github.com/CL2-UWaterloo/GenPlan.

Related papers

• Hindsight Planner: A Closed-Loop Few-Shot Planner for Embodied Instruction Following [62.10809033451526]
  This work focuses on building a task planner for Embodied Instruction Following (EIF) using Large Language Models (LLMs) We frame the task as a Partially Observable Markov Decision Process (POMDP) and aim to develop a robust planner under a few-shot assumption. Our experiments on the ALFRED dataset indicate that our planner achieves competitive performance under a few-shot assumption.
  arXiv  Detail & Related papers  (2024-12-27T10:05:45Z)
• Adaptformer: Sequence models as adaptive iterative planners [0.0]
  Decision-making in multi-task missions is a challenging problem for autonomous systems. We propose Adaptformer, an adaptive planner that utilizes sequence models for sample-efficient exploration and exploitation. We show that Adaptformer outperforms the state-of-the-art method by up to 25% in multi-goal maze reachability tasks.
  arXiv  Detail & Related papers  (2024-11-30T00:34:41Z)
• Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling [23.62433580021779]
  We advocate a self-refining scheme that iteratively refines a draft plan until an equilibrium is reached. A nested equilibrium sequence modeling procedure is devised for efficient closed-loop planning. Our method is evaluated on the VirtualHome-Env benchmark, showing advanced performance with better scaling for inference.
  arXiv  Detail & Related papers  (2024-10-02T11:42:49Z)
• AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation [81.32722475387364]
  Large Language Model-based agents have garnered significant attention and are becoming increasingly popular. Planning ability is a crucial component of an LLM-based agent, which generally entails achieving a desired goal from an initial state. Recent studies have demonstrated that utilizing expert-level trajectory for instruction-tuning LLMs effectively enhances their planning capabilities.
  arXiv  Detail & Related papers  (2024-08-01T17:59:46Z)
• Task and Motion Planning for Execution in the Real [24.01204729304763]
  This work generates task and motion plans that include actions cannot be fully grounded at planning time. Execution combines offline planned motions and online behaviors till reaching the task goal. Forty real-robot trials and motivating demonstrations are performed to evaluate the proposed framework. Results show faster execution time, less number of actions, and more success in problems where diverse gaps arise.
  arXiv  Detail & Related papers  (2024-06-05T22:30:40Z)
• Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty [56.30846158280031]
  Task planning for embodied AI has been one of the most challenging problems. We propose a task-agnostic method named 'planning as in-painting' The proposed framework achieves promising performances in various embodied AI tasks.
  arXiv  Detail & Related papers  (2023-12-02T10:07:17Z)
• AdaPlanner: Adaptive Planning from Feedback with Language Models [56.367020818139665]
  Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks. We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback. To mitigate hallucination, we develop a code-style LLM prompt structure that facilitates plan generation across a variety of tasks, environments, and agent capabilities.
  arXiv  Detail & Related papers  (2023-05-26T05:52:27Z)
• POMRL: No-Regret Learning-to-Plan with Increasing Horizons [43.693739167594295]
  We study the problem of planning under model uncertainty in an online meta-reinforcement learning setting. We propose an algorithm to meta-learn the underlying structure across tasks, utilize it to plan in each task, and upper-bound the regret of the planning loss.
  arXiv  Detail & Related papers  (2022-12-30T03:09:45Z)
• Adaptive Procedural Task Generation for Hard-Exploration Problems [78.20918366839399]
  We introduce Adaptive Procedural Task Generation (APT-Gen) to facilitate reinforcement learning in hard-exploration problems. At the heart of our approach is a task generator that learns to create tasks from a parameterized task space via a black-box procedural generation module. To enable curriculum learning in the absence of a direct indicator of learning progress, we propose to train the task generator by balancing the agent's performance in the generated tasks and the similarity to the target tasks.
  arXiv  Detail & Related papers  (2020-07-01T09:38:51Z)
• Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraints [52.58352707495122]
  We present a multi-robot allocation algorithm that decouples the key computational challenges of sequential decision-making under uncertainty and multi-agent coordination. We validate our results over a wide range of simulations on two distinct domains: multi-arm conveyor belt pick-and-place and multi-drone delivery dispatch in a city.
  arXiv  Detail & Related papers  (2020-05-27T01:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.