Self-directed Learning of Action Models using Exploratory Planning

• URL: http://arxiv.org/abs/2203.03485v1
• Date: Mon, 7 Mar 2022 15:57:10 GMT
• Title: Self-directed Learning of Action Models using Exploratory Planning
• Authors: Dustin Dannenhauer, Matthew Molineaux, Michael W. Floyd, Noah Reifsnyder, David W. Aha
• Abstract summary: We describe a novel exploratory planning agent that is capable of learning action preconditions and effects without expert traces or a given goal. The contributions of this work include a new representation for contexts called Lifted Linked Clauses, a novel exploration action selection approach using these clauses, and an empirical evaluation in a scenario from an exploration-focused video game.
• Score: 6.796748304066826
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Complex, real-world domains may not be fully modeled for an agent, especially if the agent has never operated in the domain before. The agent's ability to effectively plan and act in such a domain is influenced by its knowledge of when it can perform specific actions and the effects of those actions. We describe a novel exploratory planning agent that is capable of learning action preconditions and effects without expert traces or a given goal. The agent's architecture allows it to perform both exploratory actions as well as goal-directed actions, which opens up important considerations for how exploratory planning and goal planning should be controlled, as well as how the agent's behavior should be explained to any teammates it may have. The contributions of this work include a new representation for contexts called Lifted Linked Clauses, a novel exploration action selection approach using these clauses, an exploration planner that uses lifted linked clauses as goals in order to reach new states, and an empirical evaluation in a scenario from an exploration-focused video game demonstrating that lifted linked clauses improve exploration and action model learning against non-planning baseline agents.

Related papers

• The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
  Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
  arXiv  Detail & Related papers  (2026-01-21T15:22:21Z)
• The Cognitive Bandwidth Bottleneck: Shifting Long-Horizon Agent from Planning with Actions to Planning with Schemas [56.62286434195321]
  This paper systematically studies the effectiveness of two different action representations.<n>We propose cognitive bandwidth perspective as a conceptual framework to qualitatively understand the differences.<n>We provide an actionable guide for building more capable PwS agents for better scalable autonomy.
  arXiv  Detail & Related papers  (2025-10-08T14:47:40Z)
• Active inference for action-unaware agents [0.0]
  Active inference is a formal approach to study cognition based on the notion that adaptive agents can be seen as engaging in a process of approximate Bayesian inference.<n>We show how action-unaware agents can achieve performances comparable to action-aware ones while at a severe disadvantage.
  arXiv  Detail & Related papers  (2025-08-16T12:27:51Z)
• Toward a Theory of Agents as Tool-Use Decision-Makers [89.26889709510242]
  We argue that true autonomy requires agents to be grounded in a coherent epistemic framework that governs what they know, what they need to know, and how to acquire that knowledge efficiently.<n>We propose a unified theory that treats internal reasoning and external actions as equivalent epistemic tools, enabling agents to systematically coordinate introspection and interaction.<n>This perspective shifts the design of agents from mere action executors to knowledge-driven intelligence systems, offering a principled path toward building foundation agents capable of adaptive, efficient, and goal-directed behavior.
  arXiv  Detail & Related papers  (2025-06-01T07:52:16Z)
• Transfer Reinforcement Learning in Heterogeneous Action Spaces using Subgoal Mapping [9.81076530822611]
  We propose a method that learns a subgoal mapping between the expert agent policy and the learner agent policy. We learn this subgoal mapping by training a Long Short Term Memory (LSTM) network for a distribution of tasks. We demonstrate that the proposed learning scheme can effectively find the subgoal mapping underlying the given distribution of tasks.
  arXiv  Detail & Related papers  (2024-10-18T14:08:41Z)
• Rejecting Hallucinated State Targets during Planning [84.179112256683]
  This work first categorizes and investigates the properties of several kinds of infeasible targets.<n>We devise a strategy to reject infeasible targets with a generic target evaluator.<n>We highlight that, without proper design, the evaluator can produce delusional estimates, rendering the strategy futile.
  arXiv  Detail & Related papers  (2024-10-09T17:35:25Z)
• Ask-before-Plan: Proactive Language Agents for Real-World Planning [68.08024918064503]
  Proactive Agent Planning requires language agents to predict clarification needs based on user-agent conversation and agent-environment interaction. We propose a novel multi-agent framework, Clarification-Execution-Planning (textttCEP), which consists of three agents specialized in clarification, execution, and planning.
  arXiv  Detail & Related papers  (2024-06-18T14:07:28Z)
• Embodied Instruction Following in Unknown Environments [66.60163202450954]
  We propose an embodied instruction following (EIF) method for complex tasks in the unknown environment. We build a hierarchical embodied instruction following framework including the high-level task planner and the low-level exploration controller. For the task planner, we generate the feasible step-by-step plans for human goal accomplishment according to the task completion process and the known visual clues.
  arXiv  Detail & Related papers  (2024-06-17T17:55:40Z)
• KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents [54.09074527006576]
  Large Language Models (LLMs) have demonstrated great potential in complex reasoning tasks, yet they fall short when tackling more sophisticated challenges. This inadequacy primarily stems from the lack of built-in action knowledge in language agents. We introduce KnowAgent, a novel approach designed to enhance the planning capabilities of LLMs by incorporating explicit action knowledge.
  arXiv  Detail & Related papers  (2024-03-05T16:39:12Z)
• AI planning in the imagination: High-level planning on learned abstract search spaces [68.75684174531962]
  We propose a new method, called PiZero, that gives an agent the ability to plan in an abstract search space that the agent learns during training. We evaluate our method on multiple domains, including the traveling salesman problem, Sokoban, 2048, the facility location problem, and Pacman.
  arXiv  Detail & Related papers  (2023-08-16T22:47:16Z)
• Adaptation and Communication in Human-Robot Teaming to Handle Discrepancies in Agents' Beliefs about Plans [13.637799815698559]
  We provide an online execution algorithm based on Monte Carlo Tree Search for the agent to plan its action. We show that our agent is better equipped to work in teams without the guarantee of a shared mental model.
  arXiv  Detail & Related papers  (2023-07-07T03:05:34Z)
• Abstraction of Nondeterministic Situation Calculus Action Theories -- Extended Version [23.24285208243607]
  We develop a general framework for abstracting the behavior of an agent that operates in a nondeterministic domain. We assume that we have both an abstract and a concrete nondeterministic basic action theory. We show that if the agent has a (strong FOND) plan/strategy to achieve a goal/complete a task at the abstract level, and it can always execute the nondeterministic abstract actions to completion at the concrete level.
  arXiv  Detail & Related papers  (2023-05-20T05:42:38Z)
• Learning to Generate All Feasible Actions [4.333208181196761]
  We introduce action mapping, a novel approach that divides the learning process into two steps: first learn feasibility and subsequently, the objective. This paper focuses on the feasibility part by learning to generate all feasible actions through self-supervised querying of the feasibility model. We demonstrate the agent's proficiency in generating actions across disconnected feasible action sets.
  arXiv  Detail & Related papers  (2023-01-26T23:15:51Z)
• Online Grounding of PDDL Domains by Acting and Sensing in Unknown Environments [62.11612385360421]
  This paper proposes a framework that allows an agent to perform different tasks. We integrate machine learning models to abstract the sensory data, symbolic planning for goal achievement and path planning for navigation. We evaluate the proposed method in accurate simulated environments, where the sensors are RGB-D on-board camera, GPS and compass.
  arXiv  Detail & Related papers  (2021-12-18T21:48:20Z)
• What can I do here? A Theory of Affordances in Reinforcement Learning [65.70524105802156]
  We develop a theory of affordances for agents who learn and plan in Markov Decision Processes. Affordances play a dual role in this case, by reducing the number of actions available in any given situation. We propose an approach to learn affordances and use it to estimate transition models that are simpler and generalize better.
  arXiv  Detail & Related papers  (2020-06-26T16:34:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.