On Tackling Complex Tasks with Reward Machines and Signal Temporal Logics

• URL: http://arxiv.org/abs/2604.14440v1
• Date: Thu, 09 Apr 2026 13:39:12 GMT
• Title: On Tackling Complex Tasks with Reward Machines and Signal Temporal Logics
• Abstract summary: We propose a Reinforcement Learning (RL) based control design framework for handling complex tasks.<n>The approach extends the concept of Reward Machines (RM) with Signal Temporal Logic (STL) formulas that can be used for event generation.
• Score: 42.04219310123725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a Reinforcement Learning (RL) based control design framework for handling complex tasks. The approach extends the concept of Reward Machines (RM) with Signal Temporal Logic (STL) formulas that can be used for event generation. The use of STL allows not only a more efficient representation of rewards for complex tasks but also guiding the training process to converge towards behaviors satisfying specified requirements. We also propose an implementation of the framework that leverages the STL online monitoring algorithms. We illustrate the framework with three case studies (minigrid, cart-pole and high-way environments) with non-trivial tasks.

Related papers

• An Operator-Based Approach to STL [10.103844123344057]
  We propose a novel approach to Signal Temporal Logic (STL) based on an operator acting on reachability value functions.<n>This constitutes a new theoretical framework for handling complex multi-nested formulae.<n>In contrast to focusing on the design of STL-based reachability functions, we develop operator-based nesting rules.
  arXiv  Detail & Related papers  (2026-05-27T07:47:45Z)
• Small Model as Master Orchestrator: Learning Unified Agent-Tool Orchestration with Parallel Subtask Decomposition [61.291733522717415]
  Agent-as-Tool is a unified parallel orchestration paradigm that mitigates both agents and tools into a standardized, learnable action space.<n>ParaManager decouples planning decisions from subtask solving, enabling state-aware parallel subtask decomposition, delegation, and asynchronous execution.<n>Experiments show that ParaManager achieves strong performance across multiple benchmarks and exhibits robust generalization under unseen model pools.
  arXiv  Detail & Related papers  (2026-04-18T14:41:27Z)
• Zero-Shot Instruction Following in RL via Structured LTL Representations [50.41415009303967]
  We study instruction following in multi-task reinforcement learning, where an agent must zero-shot execute novel tasks not seen during training.<n>In this setting, linear temporal logic has recently been adopted as a powerful framework for specifying structured, temporally extended tasks.<n>While existing approaches successfully train generalist policies, they often struggle to effectively capture the rich logical and temporal structure inherent in specifications.
  arXiv  Detail & Related papers  (2026-02-15T23:22:50Z)
• Sample-Efficient Neurosymbolic Deep Reinforcement Learning [49.60927398960061]
  We propose a neuro-symbolic Deep RL approach that integrates background symbolic knowledge to improve sample efficiency.<n>Online reasoning is performed to guide the training process through two mechanisms.<n>We show improved performance over a state-of-the-art reward machine baseline.
  arXiv  Detail & Related papers  (2026-01-06T09:28:53Z)
• Zero-Shot Instruction Following in RL via Structured LTL Representations [54.08661695738909]
  Linear temporal logic (LTL) is a compelling framework for specifying complex, structured tasks for reinforcement learning (RL) agents.<n>Recent work has shown that interpreting instructions as finite automata, which can be seen as high-level programs monitoring task progress, enables learning a single generalist policy capable of executing arbitrary instructions at test time.<n>We propose a novel approach to learning a multi-task policy for following arbitrary instructions that addresses this shortcoming.
  arXiv  Detail & Related papers  (2025-12-02T10:44:51Z)
• Zero-Shot Trajectory Planning for Signal Temporal Logic Tasks [7.389002274709231]
  Signal Temporal Logic (STL) is a powerful specification language for describing complex temporal behaviors of continuous signals.<n> generating executable STL plans for STL tasks is challenging, as it requires consideration of the coupling between the task specification and the system dynamics.<n>We propose a hierarchical planning framework that enables zero-shot generalization to new STL tasks by leveraging only task-agnostic trajectory data during offline training.
  arXiv  Detail & Related papers  (2025-01-23T08:15:52Z)
• Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agents [9.529492371336286]
  Reinforcement Learning (RL) has made significant strides in enabling artificial agents to learn diverse behaviors. We propose a novel approach, called Logical Specifications-guided Dynamic Task Sampling (LSTS) LSTS learns a set of RL policies to guide an agent from an initial state to a goal state based on a high-level task specification.
  arXiv  Detail & Related papers  (2024-02-06T04:00:21Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learning [0.0]
  We propose a tractable reinforcement learning algorithm to learn a controller that enforces Signal Temporal Logic (STL) specifications. We demonstrate the utility of our approach on several STL tasks using different environments.
  arXiv  Detail & Related papers  (2022-11-30T19:38:21Z)
• Certified Reinforcement Learning with Logic Guidance [78.2286146954051]
  We propose a model-free RL algorithm that enables the use of Linear Temporal Logic (LTL) to formulate a goal for unknown continuous-state/action Markov Decision Processes (MDPs) The algorithm is guaranteed to synthesise a control policy whose traces satisfy the specification with maximal probability.
  arXiv  Detail & Related papers  (2019-02-02T20:09:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.