On Policy Reuse: An Expressive Language for Representing and Executing General Policies that Call Other Policies

• URL: http://arxiv.org/abs/2403.16824v1
• Date: Mon, 25 Mar 2024 14:48:54 GMT
• Title: On Policy Reuse: An Expressive Language for Representing and Executing General Policies that Call Other Policies
• Authors: Blai Bonet, Dominik Drexler, Hector Geffner,
• Abstract summary: A simple but powerful language has been introduced in terms of rules defined over a set of numerical features. We consider three extensions to this language aimed at making policies and sketches more flexible and reusable. The expressive power of the resulting language for policies and sketches is illustrated through a number of examples.
• Score: 14.591568801450496
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, a simple but powerful language for expressing and learning general policies and problem decompositions (sketches) has been introduced in terms of rules defined over a set of Boolean and numerical features. In this work, we consider three extensions of this language aimed at making policies and sketches more flexible and reusable: internal memory states, as in finite state controllers; indexical features, whose values are a function of the state and a number of internal registers that can be loaded with objects; and modules that wrap up policies and sketches and allow them to call each other by passing parameters. In addition, unlike general policies that select state transitions rather than ground actions, the new language allows for the selection of such actions. The expressive power of the resulting language for policies and sketches is illustrated through a number of examples.

Related papers

• Synthesizing Access Control Policies using Large Language Models [0.5762345156477738]
  Cloud compute systems allow administrators to write access control policies that govern access to private data. While policies are written in convenient languages, such as AWS Identity and Access Management Policy Language, manually written policies often become complex and error prone. In this paper, we investigate whether and how well Large Language Models (LLMs) can be used to synthesize access control policies.
  arXiv  Detail & Related papers  (2025-03-14T16:40:25Z)
• Few-shot Policy (de)composition in Conversational Question Answering [54.259440408606515]
  We propose a neuro-symbolic framework to detect policy compliance using large language models (LLMs) in a few-shot setting. We show that our approach soundly reasons about policy compliance conversations by extracting sub-questions to be answered, assigning truth values from contextual information, and explicitly producing a set of logic statements from the given policies. We apply this approach to the popular PCD and conversational machine reading benchmark, ShARC, and show competitive performance with no task-specific finetuning.
  arXiv  Detail & Related papers  (2025-01-20T08:40:15Z)
• Explainable Finite-Memory Policies for Partially Observable Markov Decision Processes [1.0499611180329806]
  Partially Observable Markov Decision Processes (POMDPs) are a fundamental framework for decision-making under uncertainty and partial observability. We provide a representation of such policies, both (i) in an interpretable formalism and (ii) typically of smaller size, together yielding higher explainability.
  arXiv  Detail & Related papers  (2024-11-20T14:42:23Z)
• PolicyLR: A Logic Representation For Privacy Policies [34.73520882451813]
  We propose PolicyLR, a new paradigm that offers a comprehensive machine-readable representation of privacy policies. PolicyLR converts privacy policies into a machine-readable format using valuations of atomic formulae. We demonstrate PolicyLR in three privacy tasks: Policy Compliance, Inconsistency Detection and Privacy Comparison Shopping.
  arXiv  Detail & Related papers  (2024-08-27T07:27:16Z)
• Learning with Language-Guided State Abstractions [58.199148890064826]
  Generalizable policy learning in high-dimensional observation spaces is facilitated by well-designed state representations. Our method, LGA, uses a combination of natural language supervision and background knowledge from language models to automatically build state representations tailored to unseen tasks. Experiments on simulated robotic tasks show that LGA yields state abstractions similar to those designed by humans, but in a fraction of the time.
  arXiv  Detail & Related papers  (2024-02-28T23:57:04Z)
• Invariant Causal Imitation Learning for Generalizable Policies [87.51882102248395]
  We propose Invariant Causal Learning (ICIL) to learn an imitation policy. ICIL learns a representation of causal features that is disentangled from the specific representations of noise variables. We show that ICIL is effective in learning imitation policies capable of generalizing to unseen environments.
  arXiv  Detail & Related papers  (2023-11-02T16:52:36Z)
• Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control [58.06223121654735]
  We show a method that taps into joint image- and goal- conditioned policies with language using only a small amount of language data. Our method achieves robust performance in the real world by learning an embedding from the labeled data that aligns language not to the goal image. We show instruction following across a variety of manipulation tasks in different scenes, with generalization to language instructions outside of the labeled data.
  arXiv  Detail & Related papers  (2023-06-30T20:09:39Z)
• Natural Language Processing for Policymaking [34.93331735602826]
  Natural language processing (NLP) uses computational tools to parse text into key information needed for policymaking. We introduce common methods of NLP, including text classification, topic modeling, event extraction, and text scaling. We highlight some potential limitations and ethical concerns when using NLP for policymaking.
  arXiv  Detail & Related papers  (2023-02-07T14:34:39Z)
• PADL: Language-Directed Physics-Based Character Control [66.517142635815]
  We present PADL, which allows users to issue natural language commands for specifying high-level tasks and low-level skills that a character should perform. We show that our framework can be applied to effectively direct a simulated humanoid character to perform a diverse array of complex motor skills.
  arXiv  Detail & Related papers  (2023-01-31T18:59:22Z)
• Pre-Trained Language Models for Interactive Decision-Making [72.77825666035203]
  We describe a framework for imitation learning in which goals and observations are represented as a sequence of embeddings. We demonstrate that this framework enables effective generalization across different environments. For test tasks involving novel goals or novel scenes, initializing policies with language models improves task completion rates by 43.6%.
  arXiv  Detail & Related papers  (2022-02-03T18:55:52Z)
• Learning Invariable Semantical Representation from Language for Extensible Policy Generalization [4.457682773596843]
  We propose a method to learn semantically invariant representations called element randomization. We theoretically prove the feasibility of learning semantically invariant representations through randomization. Experiments on challenging long-horizon tasks show that our low-level policy reliably generalizes to tasks against environment changes.
  arXiv  Detail & Related papers  (2022-01-26T08:04:27Z)
• Expressing and Exploiting the Common Subgoal Structure of Classical Planning Domains Using Sketches: Extended Version [17.63517562327928]
  We use a simple but powerful language for expressing problem decompositions introduced recently by Bonet and Geffner, called policy sketches. A policy sketch R consists of a set of Boolean and numerical features and a set of sketch rules that express how the values of these features are supposed to change. We show that many planning domains that cannot be solved by SIW are provably solvable in low time with the SIW_R algorithm.
  arXiv  Detail & Related papers  (2021-05-10T10:36:18Z)
• Policy Evaluation Networks [50.53250641051648]
  We introduce a scalable, differentiable fingerprinting mechanism that retains essential policy information in a concise embedding. Our empirical results demonstrate that combining these three elements can produce policies that outperform those that generated the training data.
  arXiv  Detail & Related papers  (2020-02-26T23:00:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.