LTL2Action: Generalizing LTL Instructions for Multi-Task RL

• URL: http://arxiv.org/abs/2102.06858v1
• Date: Sat, 13 Feb 2021 04:05:46 GMT
• Title: LTL2Action: Generalizing LTL Instructions for Multi-Task RL
• Authors: Pashootan Vaezipoor, Andrew Li, Rodrigo Toro Icarte, Sheila McIlraith
• Abstract summary: We address the problem of teaching a deep reinforcement learning (RL) agent to follow instructions in multi-task environments. We employ a well-known formal language -- linear temporal logic (LTL) -- to specify instructions, using a domain-specific vocabulary.
• Score: 4.245018630914216
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We address the problem of teaching a deep reinforcement learning (RL) agent to follow instructions in multi-task environments. We employ a well-known formal language -- linear temporal logic (LTL) -- to specify instructions, using a domain-specific vocabulary. We propose a novel approach to learning that exploits the compositional syntax and the semantics of LTL, enabling our RL agent to learn task-conditioned policies that generalize to new instructions, not observed during training. The expressive power of LTL supports the specification of a diversity of complex temporally extended behaviours that include conditionals and alternative realizations. Experiments on discrete and continuous domains demonstrate the strength of our approach in learning to solve (unseen) tasks, given LTL instructions.

Related papers

• Continual LLaVA: Continual Instruction Tuning in Large Vision-Language Models [93.5327725085853]
  Continual LLaVA is a rehearsal-free method tailored for continual instruction tuning in LVLMs. Experiments indicate that the proposed Continual LLaVA outperforms previous methods by significantly reducing the forgetting during the continual instruction tuning process.
  arXiv  Detail & Related papers  (2024-11-04T19:55:32Z)
• Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models [22.676688441884465]
  Fine-tuning pre-trained large language models (LLMs) on a diverse array of tasks has become a common approach for building models. This study investigates the task-specific information encoded in pre-trained LLMs and the effects of instruction tuning on their representations.
  arXiv  Detail & Related papers  (2024-10-25T23:38:28Z)
• DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications [59.01527054553122]
  Linear temporal logic (LTL) has recently been adopted as a powerful formalism for specifying complex, temporally extended tasks in reinforcement learning (RL) Existing approaches suffer from several shortcomings: they are often only applicable to finite-horizon fragments, are restricted to suboptimal solutions, and do not adequately handle safety constraints. In this work, we propose a novel learning approach to address these concerns. Our method leverages the structure of B"uchia, which explicitly represent the semantics of automat- specifications, to learn policies conditioned on sequences of truth assignments that lead to satisfying the desired formulae.
  arXiv  Detail & Related papers  (2024-10-06T21:30:38Z)
• Neurosymbolic AI for Enhancing Instructability in Generative AI [7.4348066967005275]
  Generative AI has transformed content creation across text, images, and music, showcasing capabilities in following instructions through prompting. This article explores why neurosymbolic AI offers a better path to enhance the instructability of Large Language Models (LLMs) We show that neurosymbolic approach enhances the reliability and context-awareness of task execution, enabling LLMs to dynamically interpret and respond to a wider range of instructional contexts with greater precision and flexibility.
  arXiv  Detail & Related papers  (2024-07-26T13:15:50Z)
• On Conditional and Compositional Language Model Differentiable Prompting [75.76546041094436]
  Prompts have been shown to be an effective method to adapt a frozen Pretrained Language Model (PLM) to perform well on downstream tasks. We propose a new model, Prompt Production System (PRopS), which learns to transform task instructions or input metadata, into continuous prompts.
  arXiv  Detail & Related papers  (2023-07-04T02:47:42Z)
• Natural Language-conditioned Reinforcement Learning with Inside-out Task Language Development and Translation [14.176720914723127]
  Natural Language-conditioned reinforcement learning (RL) enables the agents to follow human instructions. Previous approaches generally implemented language-conditioned RL by providing human instructions in natural language (NL) and training a following policy. We develop an inside-out scheme for natural language-conditioned RL by developing a task language (TL) that is task-related and unique.
  arXiv  Detail & Related papers  (2023-02-18T15:49:09Z)
• Generalizing LTL Instructions via Future Dependent Options [7.8578244861940725]
  This paper proposes a novel multi-task algorithm with improved learning efficiency and optimality. In order to propagate the rewards of satisfying future subgoals back more efficiently, we propose to train a multi-step function conditioned on the subgoal sequence. In experiments on three different domains, we evaluate the generalization capability of the agent trained by the proposed algorithm.
  arXiv  Detail & Related papers  (2022-12-08T21:44:18Z)
• Interactive Learning from Natural Language and Demonstrations using Signal Temporal Logic [5.88797764615148]
  Natural language (NL) is ambiguous, real world tasks and their safety requirements need to be communicated unambiguously. Signal Temporal Logic (STL) is a formal logic that can serve as a versatile, expressive, and unambiguous formal language to describe robotic tasks. We propose DIALOGUESTL, an interactive approach for learning correct and concise STL formulas from (often) ambiguous NL descriptions.
  arXiv  Detail & Related papers  (2022-07-01T19:08:43Z)
• Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigation [172.15808300686584]
  We describe an approach that learns the two tasks simultaneously and exploits their intrinsic correlations to boost the training of each. Our approach improves the performance of various follower models and produces accurate navigation instructions.
  arXiv  Detail & Related papers  (2022-03-30T18:15:26Z)
• LISA: Learning Interpretable Skill Abstractions from Language [85.20587800593293]
  We propose a hierarchical imitation learning framework that can learn diverse, interpretable skills from language-conditioned demonstrations. Our method demonstrates a more natural way to condition on language in sequential decision-making problems.
  arXiv  Detail & Related papers  (2022-02-28T19:43:24Z)
• Contrastive Instruction-Trajectory Learning for Vision-Language Navigation [66.16980504844233]
  A vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction. Previous works fail to discriminate the similarities and discrepancies across instruction-trajectory pairs and ignore the temporal continuity of sub-instructions. We propose a Contrastive Instruction-Trajectory Learning framework that explores invariance across similar data samples and variance across different ones to learn distinctive representations for robust navigation.
  arXiv  Detail & Related papers  (2021-12-08T06:32:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.