Abstracting Situation Calculus Action Theories

• URL: http://arxiv.org/abs/2410.14712v1
• Date: Wed, 09 Oct 2024 16:34:28 GMT
• Title: Abstracting Situation Calculus Action Theories
• Authors: Bita Banihashemi, Giuseppe De Giacomo, Yves Lespérance,
• Abstract summary: We assume that we have a high-level specification and a low-level specification of the agent, both represented as basic action theories. A refinement mapping specifies how each high-level action is implemented by a low-level ConGolog program. We identify a set of basic action theory constraints that ensure that for any low-level action sequence, there is a unique high-level action sequence.
• Score: 24.181367387692944
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We develop a general framework for agent abstraction based on the situation calculus and the ConGolog agent programming language. We assume that we have a high-level specification and a low-level specification of the agent, both represented as basic action theories. A refinement mapping specifies how each high-level action is implemented by a low-level ConGolog program and how each high-level fluent can be translated into a low-level formula. We define a notion of sound abstraction between such action theories in terms of the existence of a suitable bisimulation between their respective models. Sound abstractions have many useful properties that ensure that we can reason about the agent's actions (e.g., executability, projection, and planning) at the abstract level, and refine and concretely execute them at the low level. We also characterize the notion of complete abstraction where all actions (including exogenous ones) that the high level thinks can happen can in fact occur at the low level. To facilitate verifying that one has a sound/complete abstraction relative to a mapping, we provide a set of necessary and sufficient conditions. Finally, we identify a set of basic action theory constraints that ensure that for any low-level action sequence, there is a unique high-level action sequence that it refines. This allows us to track/monitor what the low-level agent is doing and describe it in abstract terms (i.e., provide high-level explanations, for instance, to a client or manager).

Related papers

• Structured Agent Distillation for Large Language Model [58.22497891295258]
  We propose Structured Agent Distillation, a framework that compresses large LLM-based agents into smaller student models.<n>Our method segments trajectories into [REASON] and [ACT] spans, applying segment-specific losses to align each component with the teacher's behavior.<n>Experiments on ALFWorld, HotPotQA-ReAct, and WebShop show that our approach consistently outperforms token-level and imitation learning baselines.
  arXiv  Detail & Related papers  (2025-05-20T02:01:55Z)
• Composing Reinforcement Learning Policies, with Formal Guarantees [15.690880632229202]
  We propose a novel framework to controller design in environments with a two-level structure. The framework "separates concerns" by using different design techniques for low- and high-level tasks.
  arXiv  Detail & Related papers  (2024-02-21T13:10:58Z)
• Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning [63.58935783293342]
  Causal Bisimulation Modeling (CBM) is a method that learns the causal relationships in the dynamics and reward functions for each task to derive a minimal, task-specific abstraction. CBM's learned implicit dynamics models identify the underlying causal relationships and state abstractions more accurately than explicit ones.
  arXiv  Detail & Related papers  (2024-01-23T05:43:15Z)
• AI planning in the imagination: High-level planning on learned abstract search spaces [68.75684174531962]
  We propose a new method, called PiZero, that gives an agent the ability to plan in an abstract search space that the agent learns during training. We evaluate our method on multiple domains, including the traveling salesman problem, Sokoban, 2048, the facility location problem, and Pacman.
  arXiv  Detail & Related papers  (2023-08-16T22:47:16Z)
• Abstraction of Nondeterministic Situation Calculus Action Theories -- Extended Version [23.24285208243607]
  We develop a general framework for abstracting the behavior of an agent that operates in a nondeterministic domain. We assume that we have both an abstract and a concrete nondeterministic basic action theory. We show that if the agent has a (strong FOND) plan/strategy to achieve a goal/complete a task at the abstract level, and it can always execute the nondeterministic abstract actions to completion at the concrete level.
  arXiv  Detail & Related papers  (2023-05-20T05:42:38Z)
• Abstracting Noisy Robot Programs [17.04153879817609]
  We describe an approach to abstraction of probabilistic and dynamic systems. Based on a variant of the situation calculus with probabilistic belief, we define a notion of bisimulation. We obtain abstract Golog programs that omit unnecessary details and which can be translated back to a detailed program for actual execution.
  arXiv  Detail & Related papers  (2022-04-07T16:04:19Z)
• Inventing Relational State and Action Abstractions for Effective and Efficient Bilevel Planning [26.715198108255162]
  We develop a novel framework for learning state and action abstractions. We learn relational, neuro-symbolic abstractions that generalize over object identities and numbers. We show that our learned abstractions are able to quickly solve held-out tasks of longer horizons.
  arXiv  Detail & Related papers  (2022-03-17T22:13:09Z)
• Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization [30.456180468318305]
  In the sequential decision making setting, an agent aims to achieve systematic generalization over a large, possibly infinite, set of environments. In this paper, we provide a tractable formulation of systematic generalization by employing a causal viewpoint. Under specific structural assumptions, we provide a simple learning algorithm that guarantees any desired planning error up to an unavoidable sub-optimality term.
  arXiv  Detail & Related papers  (2022-02-14T08:34:51Z)
• Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents [111.33545170562337]
  We investigate the possibility of grounding high-level tasks, expressed in natural language, to a chosen set of actionable steps. We find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into low-level plans. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions.
  arXiv  Detail & Related papers  (2022-01-18T18:59:45Z)
• Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning [120.38381203153159]
  Reinforcement learning can train policies that effectively perform complex tasks. For long-horizon tasks, the performance of these methods degrades with horizon, often necessitating reasoning over and composing lower-level skills. We propose Value Function Spaces: a simple approach that produces such a representation by using the value functions corresponding to each lower-level skill.
  arXiv  Detail & Related papers  (2021-11-04T22:46:16Z)
• Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language [81.73820295186727]
  We propose a formalism of procedures as programs, a powerful yet intuitive method of representing hierarchical procedural knowledge for agent command and control. We instantiate this framework on the IQA and ALFRED datasets for NL instruction following.
  arXiv  Detail & Related papers  (2021-09-16T20:36:21Z)
• What can I do here? A Theory of Affordances in Reinforcement Learning [65.70524105802156]
  We develop a theory of affordances for agents who learn and plan in Markov Decision Processes. Affordances play a dual role in this case, by reducing the number of actions available in any given situation. We propose an approach to learn affordances and use it to estimate transition models that are simpler and generalize better.
  arXiv  Detail & Related papers  (2020-06-26T16:34:53Z)
• From proprioception to long-horizon planning in novel environments: A hierarchical RL model [4.44317046648898]
  In this work, we introduce a simple, three-level hierarchical architecture that reflects different types of reasoning. We apply our method to a series of navigation tasks in the Mujoco Ant environment.
  arXiv  Detail & Related papers  (2020-06-11T17:19:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.