A Reward-Petri-Net Interpretation of Temporal Behavior Trees

• URL: http://arxiv.org/abs/2606.21350v1
• Date: Fri, 19 Jun 2026 11:47:54 GMT
• Title: A Reward-Petri-Net Interpretation of Temporal Behavior Trees
• Abstract summary: This paper introduces an interpretation of Temporal Behavior Trees (TBTs) as Reward-Petri-Nets (RPNs) for reinforcement learning (RL)
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces an interpretation of Temporal Behavior Trees (TBTs) as Reward-Petri-Nets (RPNs) for reinforcement learning (RL). Designing reward functions for complex, long-horizon robotic tasks is notoriously difficult, especially when tasks have hierarchical structure and temporal constraints. TBTs extend conventional behavior trees (BTs) used in robotic applications by incorporating temporal properties into their leaf nodes. This allows TBTs to represents not only the behavioral task structure defined by BT operators such as Sequence, Fallback, and Parallel, but also the task's temporal constraints. In this work, the constraints are specified in the leaf nodes using Linear Temporal Logic. In order to inform RL rewards using TBTs, we provide a translation from TBT into a Petri Net (PN) and show how rewards can be automatically assigned based on the TBT's structure, resulting in a RPN. In a series of increasingly challenging environments, we demonstrate how TBT-based rewards enable learning where vanilla RL fails, improve sample efficiency, and offer flexible, intuitive control over the learning progress. We showcase the learning impact by using different reward distribution schemes and TBT structures.

Related papers

• Reward Shaping and Action Masking for Compositional Tasks using Behavior Trees and LLMs [3.285378336473673]
  Reinforcement learning can be used to optimize agent policies to complete subtasks.<n>Recent work uses large language models (LLMs) to automate reward shaping and action masking.<n>We develop masking reward behavior tree (MRBT), a symbolic structure used as a reactive and modular reward and action mask function.
  arXiv  Detail & Related papers  (2026-05-07T07:33:08Z)
• Zero-Shot Instruction Following in RL via Structured LTL Representations [50.41415009303967]
  We study instruction following in multi-task reinforcement learning, where an agent must zero-shot execute novel tasks not seen during training.<n>In this setting, linear temporal logic has recently been adopted as a powerful framework for specifying structured, temporally extended tasks.<n>While existing approaches successfully train generalist policies, they often struggle to effectively capture the rich logical and temporal structure inherent in specifications.
  arXiv  Detail & Related papers  (2026-02-15T23:22:50Z)
• Progress Constraints for Reinforcement Learning in Behavior Trees [4.532332159509032]
  Reinforcement Learning (RL) can learn near-optimal controllers but sometimes struggles with sparse rewards, safe exploration, and long-horizon credit assignment.<n>We propose progress constraints, a novel mechanism where feasibility estimators constrain the allowed action set based on theoretical BT convergence results.<n> Empirical evaluations in a 2D proof-of-concept and a high-fidelity warehouse environment demonstrate improved performance, sample efficiency, and constraint satisfaction.
  arXiv  Detail & Related papers  (2026-02-06T09:26:44Z)
• SRT: Accelerating Reinforcement Learning via Speculative Rollout with Tree-Structured Cache [27.164352776270217]
  Speculative Rollout with Tree-Structured Cache (SRT) is a model-free approach to accelerate on-policy reinforcement learning (RL)<n>SRT exploits the empirical similarity of rollouts for the same prompt across training steps by storing previously generated continuations in a per-prompt tree-structured cache.<n>It consistently reduces generation and step latency and lowers per-token inference cost, achieving up to 2.08x wall-clock time speedup during rollout.
  arXiv  Detail & Related papers  (2026-01-14T02:34:48Z)
• Zero-Shot Instruction Following in RL via Structured LTL Representations [54.08661695738909]
  Linear temporal logic (LTL) is a compelling framework for specifying complex, structured tasks for reinforcement learning (RL) agents.<n>Recent work has shown that interpreting instructions as finite automata, which can be seen as high-level programs monitoring task progress, enables learning a single generalist policy capable of executing arbitrary instructions at test time.<n>We propose a novel approach to learning a multi-task policy for following arbitrary instructions that addresses this shortcoming.
  arXiv  Detail & Related papers  (2025-12-02T10:44:51Z)
• Gradient Boosting Reinforcement Learning [9.66275447955737]
  Gradient Boosting Reinforcement Learning (GBRL) is a framework that adapts the strengths of gradient boosting trees (GBT) to reinforcement learning (RL) tasks.<n>We show that GBRL outperforms neural networks (NNs) in domains with structured observations and categorical features.
  arXiv  Detail & Related papers  (2024-07-11T07:52:33Z)
• Shrink-Perturb Improves Architecture Mixing during Population Based Training for Neural Architecture Search [62.997667081978825]
  We show that simultaneously training and mixing neural networks is a promising way to conduct Neural Architecture Search (NAS) We propose PBT-NAS, an adaptation of PBT to NAS where architectures are improved during training by replacing poorly-performing networks in a population with the result of mixing well-performing ones and inheriting the weights using the shrink-perturb technique.
  arXiv  Detail & Related papers  (2023-07-28T15:29:52Z)
• Scalable Real-Time Recurrent Learning Using Columnar-Constructive Networks [19.248060562241296]
  We propose two constraints that make real-time recurrent learning scalable. We show that by either decomposing the network into independent modules or learning the network in stages, we can make RTRL scale linearly with the number of parameters. We demonstrate the effectiveness of our approach over Truncated-BPTT on a prediction benchmark inspired by animal learning and by doing policy evaluation of pre-trained policies for Atari 2600 games.
  arXiv  Detail & Related papers  (2023-01-20T23:17:48Z)
• Online Training Through Time for Spiking Neural Networks [66.7744060103562]
  Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Recent progress in training methods has enabled successful deep SNNs on large-scale tasks with low latency. We propose online training through time (OTTT) for SNNs, which is derived from BPTT to enable forward-in-time learning.
  arXiv  Detail & Related papers  (2022-10-09T07:47:56Z)
• TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference [54.791572981834435]
  Existing pre-trained language models (PLMs) are often computationally expensive in inference. We propose a dynamic token reduction approach to accelerate PLMs' inference, named TR-BERT. TR-BERT formulates the token reduction process as a multi-step token selection problem and automatically learns the selection strategy via reinforcement learning.
  arXiv  Detail & Related papers  (2021-05-25T02:28:51Z)
• Towards Evaluating and Training Verifiably Robust Neural Networks [81.39994285743555]
  We study the relationship between IBP and CROWN, and prove that CROWN is always tighter than IBP when choosing appropriate bounding lines. We propose a relaxed version of CROWN, linear bound propagation (LBP), that can be used to verify large networks to obtain lower verified errors.
  arXiv  Detail & Related papers  (2021-04-01T13:03:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.