CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPs

• URL: http://arxiv.org/abs/2007.13202v3
• Date: Sun, 8 Nov 2020 00:10:55 GMT
• Title: CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPs
• Authors: Rohan Chitnis, Tom Silver, Beomjoon Kim, Leslie Pack Kaelbling, Tomas Lozano-Perez
• Abstract summary: A general meta-planning strategy is to learn to impose constraints on the states considered and actions taken by the agent. We propose a context-specific abstract Markov decision process that affords efficient planning. We find planning with learned CAMPs to consistently outperform baselines.
• Score: 32.15589254223466
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Meta-planning, or learning to guide planning from experience, is a promising approach to improving the computational cost of planning. A general meta-planning strategy is to learn to impose constraints on the states considered and actions taken by the agent. We observe that (1) imposing a constraint can induce context-specific independences that render some aspects of the domain irrelevant, and (2) an agent can take advantage of this fact by imposing constraints on its own behavior. These observations lead us to propose the context-specific abstract Markov decision process (CAMP), an abstraction of a factored MDP that affords efficient planning. We then describe how to learn constraints to impose so the CAMP optimizes a trade-off between rewards and computational cost. Our experiments consider five planners across four domains, including robotic navigation among movable obstacles (NAMO), robotic task and motion planning for sequential manipulation, and classical planning. We find planning with learned CAMPs to consistently outperform baselines, including Stilman's NAMO-specific algorithm. Video: https://youtu.be/wTXt6djcAd4 Code: https://git.io/JTnf6

Related papers

• Plan Your Travel and Travel with Your Plan: Wide-Horizon Planning and Evaluation via LLM [58.50687282180444]
  Travel planning is a complex task requiring the integration of diverse real-world information and user preferences.<n>We formulate this as an $L3$ planning problem, emphasizing long context, long instruction, and long output.<n>We introduce Multiple Aspects of Planning (MAoP), enabling LLMs to conduct wide-horizon thinking to solve complex planning problems.
  arXiv  Detail & Related papers  (2025-06-14T09:37:59Z)
• Projection Abstractions in Planning Under the Lenses of Abstractions for MDPs [1.46184883556683]
  The concept of abstraction has been independently developed both in the context of AI Planning and discounted Markov Decision Processes (MDPs) This paper aims to look at projection abstractions in Planning through the lenses of discounted MDPs. Starting from a projection abstraction built according to Classical or Probabilistic Planning techniques, we will show how the same abstraction can be obtained under the abstraction frameworks available for discounted MDPs.
  arXiv  Detail & Related papers  (2024-12-03T17:43:28Z)
• On The Planning Abilities of OpenAI's o1 Models: Feasibility, Optimality, and Generalizability [59.72892401927283]
  We evaluate the planning capabilities of OpenAI's o1 models across a variety of benchmark tasks. Our results reveal that o1-preview outperforms GPT-4 in adhering to task constraints.
  arXiv  Detail & Related papers  (2024-09-30T03:58:43Z)
• Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach [57.788675205519986]
  We learn high-quality traces from POMDP executions generated by any solver. We exploit data- and time-efficient Indu Logic Programming (ILP) to generate interpretable belief-based policy specifications. We show that learneds expressed in Answer Set Programming (ASP) yield performance superior to neural networks and similar to optimal handcrafted task-specifics within lower computational time.
  arXiv  Detail & Related papers  (2024-02-29T15:36:01Z)
• LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning [65.86754998249224]
  We develop a novel hybrid planner that leverages a conventional rule-based planner in conjunction with an LLM-based planner. Our approach navigates complex scenarios which existing planners struggle with, produces well-reasoned outputs while also remaining grounded through working alongside the rule-based approach.
  arXiv  Detail & Related papers  (2023-12-30T02:53:45Z)
• AI planning in the imagination: High-level planning on learned abstract search spaces [68.75684174531962]
  We propose a new method, called PiZero, that gives an agent the ability to plan in an abstract search space that the agent learns during training. We evaluate our method on multiple domains, including the traveling salesman problem, Sokoban, 2048, the facility location problem, and Pacman.
  arXiv  Detail & Related papers  (2023-08-16T22:47:16Z)
• DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observability [58.75803543245372]
  Task and Motion Planning (TAMP) approaches are suited for planning multi-step autonomous robot manipulation. We propose to overcome these limitations by composing diffusion models using a TAMP system. We show how the combination of classical TAMP, generative modeling, and latent embedding enables multi-step constraint-based reasoning.
  arXiv  Detail & Related papers  (2023-06-22T20:40:24Z)
• AdaPlanner: Adaptive Planning from Feedback with Language Models [56.367020818139665]
  Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks. We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback. To mitigate hallucination, we develop a code-style LLM prompt structure that facilitates plan generation across a variety of tasks, environments, and agent capabilities.
  arXiv  Detail & Related papers  (2023-05-26T05:52:27Z)
• Inventing Relational State and Action Abstractions for Effective and Efficient Bilevel Planning [26.715198108255162]
  We develop a novel framework for learning state and action abstractions. We learn relational, neuro-symbolic abstractions that generalize over object identities and numbers. We show that our learned abstractions are able to quickly solve held-out tasks of longer horizons.
  arXiv  Detail & Related papers  (2022-03-17T22:13:09Z)
• Visual scoping operations for physical assembly [0.0]
  We propose visual scoping, a strategy that interleaves planning and acting by alternately defining a spatial region as the next subgoal. We find that visual scoping achieves comparable task performance to the subgoal planner while requiring only a fraction of the total computational cost.
  arXiv  Detail & Related papers  (2021-06-10T10:50:35Z)
• Learning Symbolic Operators for Task and Motion Planning [29.639902380586253]
  integrated task and motion planners (TAMP) handle the complex interaction between motion-level decisions and task-level plan feasibility. TAMP approaches rely on domain-specific symbolic operators to guide the task-level search, making planning efficient. We propose a bottom-up relational learning method for operator learning and show how the learned operators can be used for planning in a TAMP system.
  arXiv  Detail & Related papers  (2021-02-28T19:08:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.