LTL2Action: Generalizing LTL Instructions for Multi-Task RL
- URL: http://arxiv.org/abs/2102.06858v1
- Date: Sat, 13 Feb 2021 04:05:46 GMT
- Title: LTL2Action: Generalizing LTL Instructions for Multi-Task RL
- Authors: Pashootan Vaezipoor, Andrew Li, Rodrigo Toro Icarte, Sheila McIlraith
- Abstract summary: We address the problem of teaching a deep reinforcement learning (RL) agent to follow instructions in multi-task environments.
We employ a well-known formal language -- linear temporal logic (LTL) -- to specify instructions, using a domain-specific vocabulary.
- Score: 4.245018630914216
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We address the problem of teaching a deep reinforcement learning (RL) agent
to follow instructions in multi-task environments. We employ a well-known
formal language -- linear temporal logic (LTL) -- to specify instructions,
using a domain-specific vocabulary. We propose a novel approach to learning
that exploits the compositional syntax and the semantics of LTL, enabling our
RL agent to learn task-conditioned policies that generalize to new
instructions, not observed during training. The expressive power of LTL
supports the specification of a diversity of complex temporally extended
behaviours that include conditionals and alternative realizations. Experiments
on discrete and continuous domains demonstrate the strength of our approach in
learning to solve (unseen) tasks, given LTL instructions.
Related papers
- SwitchCIT: Switching for Continual Instruction Tuning of Large Language Models [14.085371250265224]
Large language models (LLMs) have exhibited impressive capabilities in various domains, particularly in general language understanding.
However these models, trained on massive text data, may not be finely optimized for specific tasks triggered by instructions.
This work addresses the catastrophic forgetting in continual instruction learning for LLMs through a switching mechanism for routing computations to parameter-efficient tuned models.
arXiv Detail & Related papers (2024-07-16T14:37:33Z) - Controllable Navigation Instruction Generation with Chain of Thought Prompting [74.34604350917273]
We propose C-Instructor, which utilizes the chain-of-thought-style prompt for style-controllable and content-controllable instruction generation.
C-Instructor renders generated instructions more accessible to follow and offers greater controllability over the manipulation of landmark objects.
arXiv Detail & Related papers (2024-07-10T07:37:20Z) - Vision-Language Models Provide Promptable Representations for Reinforcement Learning [67.40524195671479]
We propose a novel approach that uses the vast amounts of general and indexable world knowledge encoded in vision-language models (VLMs) pre-trained on Internet-scale data for embodied reinforcement learning (RL)
We show that our approach can use chain-of-thought prompting to produce representations of common-sense semantic reasoning, improving policy performance in novel scenes by 1.5 times.
arXiv Detail & Related papers (2024-02-05T00:48:56Z) - On Conditional and Compositional Language Model Differentiable Prompting [75.76546041094436]
Prompts have been shown to be an effective method to adapt a frozen Pretrained Language Model (PLM) to perform well on downstream tasks.
We propose a new model, Prompt Production System (PRopS), which learns to transform task instructions or input metadata, into continuous prompts.
arXiv Detail & Related papers (2023-07-04T02:47:42Z) - Natural Language-conditioned Reinforcement Learning with Inside-out Task
Language Development and Translation [14.176720914723127]
Natural Language-conditioned reinforcement learning (RL) enables the agents to follow human instructions.
Previous approaches generally implemented language-conditioned RL by providing human instructions in natural language (NL) and training a following policy.
We develop an inside-out scheme for natural language-conditioned RL by developing a task language (TL) that is task-related and unique.
arXiv Detail & Related papers (2023-02-18T15:49:09Z) - Generalizing LTL Instructions via Future Dependent Options [7.8578244861940725]
This paper proposes a novel multi-task algorithm with improved learning efficiency and optimality.
In order to propagate the rewards of satisfying future subgoals back more efficiently, we propose to train a multi-step function conditioned on the subgoal sequence.
In experiments on three different domains, we evaluate the generalization capability of the agent trained by the proposed algorithm.
arXiv Detail & Related papers (2022-12-08T21:44:18Z) - Interactive Learning from Natural Language and Demonstrations using
Signal Temporal Logic [5.88797764615148]
Natural language (NL) is ambiguous, real world tasks and their safety requirements need to be communicated unambiguously.
Signal Temporal Logic (STL) is a formal logic that can serve as a versatile, expressive, and unambiguous formal language to describe robotic tasks.
We propose DIALOGUESTL, an interactive approach for learning correct and concise STL formulas from (often) ambiguous NL descriptions.
arXiv Detail & Related papers (2022-07-01T19:08:43Z) - Counterfactual Cycle-Consistent Learning for Instruction Following and
Generation in Vision-Language Navigation [172.15808300686584]
We describe an approach that learns the two tasks simultaneously and exploits their intrinsic correlations to boost the training of each.
Our approach improves the performance of various follower models and produces accurate navigation instructions.
arXiv Detail & Related papers (2022-03-30T18:15:26Z) - LISA: Learning Interpretable Skill Abstractions from Language [85.20587800593293]
We propose a hierarchical imitation learning framework that can learn diverse, interpretable skills from language-conditioned demonstrations.
Our method demonstrates a more natural way to condition on language in sequential decision-making problems.
arXiv Detail & Related papers (2022-02-28T19:43:24Z) - Contrastive Instruction-Trajectory Learning for Vision-Language
Navigation [66.16980504844233]
A vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction.
Previous works fail to discriminate the similarities and discrepancies across instruction-trajectory pairs and ignore the temporal continuity of sub-instructions.
We propose a Contrastive Instruction-Trajectory Learning framework that explores invariance across similar data samples and variance across different ones to learn distinctive representations for robust navigation.
arXiv Detail & Related papers (2021-12-08T06:32:52Z) - ELLA: Exploration through Learned Language Abstraction [6.809870486883877]
ELLA is a reward shaping approach that correlates high-level instructions with simpler low-level instructions to enrich the sparse rewards afforded by the environment.
ELLA shows a significant gain in sample efficiency across several environments compared to competitive language-based reward shaping and no-shaping methods.
arXiv Detail & Related papers (2021-03-10T02:18:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.