Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructions

• URL: http://arxiv.org/abs/2602.06746v1
• Date: Fri, 06 Feb 2026 14:46:27 GMT
• Title: Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructions
• Authors: Alessandro Abate, Giuseppe De Giacomo, Mathias Jackermeier, Jan Kretínský, Maximilian Prokop, Christoph Weinhuber,
• Abstract summary: We study multi-task reinforcement learning (RL), a setting in which an agent learns a single, universal policy.<n>We present a novel task embedding technique leveraging a new generation of semantic translations-to-automata.
• Score: 61.479946958462754
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study multi-task reinforcement learning (RL), a setting in which an agent learns a single, universal policy capable of generalising to arbitrary, possibly unseen tasks. We consider tasks specified as linear temporal logic (LTL) formulae, which are commonly used in formal methods to specify properties of systems, and have recently been successfully adopted in RL. In this setting, we present a novel task embedding technique leveraging a new generation of semantic LTL-to-automata translations, originally developed for temporal synthesis. The resulting semantically labelled automata contain rich, structured information in each state that allow us to (i) compute the automaton efficiently on-the-fly, (ii) extract expressive task embeddings used to condition the policy, and (iii) naturally support full LTL. Experimental results in a variety of domains demonstrate that our approach achieves state-of-the-art performance and is able to scale to complex specifications where existing methods fail.

Related papers

• Zero-Shot Instruction Following in RL via Structured LTL Representations [50.41415009303967]
  We study instruction following in multi-task reinforcement learning, where an agent must zero-shot execute novel tasks not seen during training.<n>In this setting, linear temporal logic has recently been adopted as a powerful framework for specifying structured, temporally extended tasks.<n>While existing approaches successfully train generalist policies, they often struggle to effectively capture the rich logical and temporal structure inherent in specifications.
  arXiv  Detail & Related papers  (2026-02-15T23:22:50Z)
• PlatoLTL: Learning to Generalize Across Symbols in LTL Instructions for Multi-Task RL [55.58188508467081]
  linear temporal logic (LTL) is a powerful formalism for specifying structured, temporally extended tasks to RL agents.<n>We present PlatoLTL, a novel approach that enables policies to zero-shot generalize not only compositionally across formula structures, but also parametrically across propositions.
  arXiv  Detail & Related papers  (2026-01-30T12:11:55Z)
• Zero-Shot Instruction Following in RL via Structured LTL Representations [54.08661695738909]
  Linear temporal logic (LTL) is a compelling framework for specifying complex, structured tasks for reinforcement learning (RL) agents.<n>Recent work has shown that interpreting instructions as finite automata, which can be seen as high-level programs monitoring task progress, enables learning a single generalist policy capable of executing arbitrary instructions at test time.<n>We propose a novel approach to learning a multi-task policy for following arbitrary instructions that addresses this shortcoming.
  arXiv  Detail & Related papers  (2025-12-02T10:44:51Z)
• Learning Task Representations from In-Context Learning [67.66042137487287]
  Large language models (LLMs) have demonstrated remarkable proficiency in in-context learning (ICL)<n>We introduce an automated formulation for encoding task information in ICL prompts as a function of attention heads.<n>The proposed method successfully extracts task-specific information from in-context demonstrations and excels in both text and regression tasks.
  arXiv  Detail & Related papers  (2025-02-08T00:16:44Z)
• Inverse Reinforcement Learning of Autonomous Behaviors Encoded as Weighted Finite Automata [18.972270182221262]
  This paper presents a method for learning logical task specifications and cost functions from demonstrations. We employ a spectral learning approach to extract a weighted finite automaton (WFA), approximating the unknown logic structure of the task. We define a product between the WFA for high-level task guidance and a Labeled Markov decision process (L-MDP) for low-level control and optimize a cost function that matches the demonstrator's behavior.
  arXiv  Detail & Related papers  (2021-03-10T06:42:10Z)
• LTL2Action: Generalizing LTL Instructions for Multi-Task RL [4.245018630914216]
  We address the problem of teaching a deep reinforcement learning (RL) agent to follow instructions in multi-task environments. We employ a well-known formal language -- linear temporal logic (LTL) -- to specify instructions, using a domain-specific vocabulary.
  arXiv  Detail & Related papers  (2021-02-13T04:05:46Z)
• Learning Optimal Strategies for Temporal Tasks in Stochastic Games [23.012106429532633]
  We introduce a model-free reinforcement learning (RL) approach to derive controllers from given specifications. We learn optimal control strategies that maximize the probability of satisfying the specifications against the worst-case environment behavior.
  arXiv  Detail & Related papers  (2021-02-08T16:10:50Z)
• Certified Reinforcement Learning with Logic Guidance [78.2286146954051]
  We propose a model-free RL algorithm that enables the use of Linear Temporal Logic (LTL) to formulate a goal for unknown continuous-state/action Markov Decision Processes (MDPs) The algorithm is guaranteed to synthesise a control policy whose traces satisfy the specification with maximal probability.
  arXiv  Detail & Related papers  (2019-02-02T20:09:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.