Learning specifications for reactive synthesis with safety constraints

• URL: http://arxiv.org/abs/2601.05533v1
• Date: Fri, 09 Jan 2026 05:17:45 GMT
• Title: Learning specifications for reactive synthesis with safety constraints
• Abstract summary: We model latent tasks as probabilistic formal languages and introduce a tailored reactive synthesis framework.<n>We adapt existing evidence-driven state algorithms merging and incorporate safety requirements throughout the learning process.<n>Our approach models the interaction as a two-player game between the robot and the environment, accounting for dynamic changes.
• Score: 9.76292270520872
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a novel approach to learning from demonstration that enables robots to autonomously execute complex tasks in dynamic environments. We model latent tasks as probabilistic formal languages and introduce a tailored reactive synthesis framework that balances robot costs with user task preferences. Our methodology focuses on safety-constrained learning and inferring formal task specifications as Probabilistic Deterministic Finite Automata (PDFA). We adapt existing evidence-driven state merging algorithms and incorporate safety requirements throughout the learning process to ensure that the learned PDFA always complies with safety constraints. Furthermore, we introduce a multi-objective reactive synthesis algorithm that generates deterministic strategies that are guaranteed to satisfy the PDFA task while optimizing the trade-offs between user preferences and robot costs, resulting in a Pareto front of optimal solutions. Our approach models the interaction as a two-player game between the robot and the environment, accounting for dynamic changes. We present a computationally-tractable value iteration algorithm to generate the Pareto front and the corresponding deterministic strategies. Comprehensive experimental results demonstrate the effectiveness of our algorithms across various robots and tasks, showing that the learned PDFA never includes unsafe behaviors and that synthesized strategies consistently achieve the task while meeting both the robot cost and user-preference requirements.

Related papers

• A Unified Multi-Layer Framework for Skill Acquisition from Imperfect Human Demonstrations [7.8260466406864575]
  Current Human-Robot Interaction (HRI) systems for skill teaching are fragmented.<n>Existing approaches in the literature do not offer a cohesive framework that is simultaneously efficient, intuitive, and universally safe.<n>This paper presents a novel, layered control framework that enables robust, compliant Learning from Demonstration.
  arXiv  Detail & Related papers  (2026-04-09T15:13:19Z)
• Interaction-Grounded Learning for Contextual Markov Decision Processes with Personalized Feedback [59.287761696290865]
  We propose a computationally efficient algorithm that achieves a sublinear regret guarantee for contextual episodic Markov Decision Processes (MDPs) with personalized feedback.<n>We demonstrate the effectiveness of our method in learning personalized objectives from multi-turn interactions through experiments on both a synthetic episodic MDP and a real-world user booking dataset.
  arXiv  Detail & Related papers  (2026-02-09T06:29:54Z)
• Action Flow Matching for Continual Robot Learning [54.10050120844738]
  Continual learning in robotics seeks systems that can constantly adapt to changing environments and tasks.<n>We introduce a generative framework leveraging flow matching for online robot dynamics model alignment.<n>We find that by transforming the actions themselves rather than exploring with a misaligned model, the robot collects informative data more efficiently.
  arXiv  Detail & Related papers  (2025-04-25T16:26:15Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer.<n>By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedback [3.10979520014442]
  An increasing level of autonomy of robots poses challenges of trust and social acceptance, especially in human-robot interaction scenarios.<n>This requires an interpretable implementation of robotic cognitive capabilities, possibly based on formal methods as logics for the definition of task specifications.<n>We propose an offline algorithm based on inductive logic programming from noisy examples to extract task specifications.
  arXiv  Detail & Related papers  (2025-01-13T17:25:46Z)
• Online Pareto-Optimal Decision-Making for Complex Tasks using Active Inference [14.470714123175972]
  This paper introduces a novel framework for multi-objective reinforcement learning. It ensures safe task execution, optimize trade-offs between objectives, and adheres to user preferences. Case studies and benchmarks on both manipulation and mobile robots show that our framework outperforms other methods.
  arXiv  Detail & Related papers  (2024-06-17T18:03:45Z)
• Co-learning Planning and Control Policies Constrained by Differentiable Logic Specifications [4.12484724941528]
  This paper presents a novel reinforcement learning approach to solving high-dimensional robot navigation tasks. We train high-quality policies with much fewer samples compared to existing reinforcement learning algorithms. Our approach also demonstrates capabilities for high-dimensional control and avoiding suboptimal policies via policy alignment.
  arXiv  Detail & Related papers  (2023-03-02T15:24:24Z)
• Strategy Discovery and Mixture in Lifelong Learning from Heterogeneous Demonstration [1.2891210250935146]
  Learning from Demonstration (LfD) approaches empower end-users to teach robots novel tasks via demonstrations of the desired behaviors. In this paper, we propose a novel algorithm, Dynamic Multi-Strategy Reward Distillation (DMSRD), which distills common knowledge between heterogeneous demonstrations. Our personalized, federated, and lifelong LfD architecture surpasses benchmarks in two continuous control problems with an average 77% improvement in policy returns and 42% improvement in log likelihood.
  arXiv  Detail & Related papers  (2022-02-14T20:10:25Z)
• Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms [60.59764170868101]
  Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. We formulate it as a few-shot meta-learning problem where the goal is to find a model that captures the common structure shared across different robotic platforms. We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots.
  arXiv  Detail & Related papers  (2021-03-05T14:16:20Z)
• Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees [63.31965375413414]
  We propose to solve multi-task problems through learning structured policies from human demonstrations. Our structured policy is inspired by RMPflow, a framework for combining subtask policies on different spaces. We derive an end-to-end learning objective function that is suitable for the multi-task problem.
  arXiv  Detail & Related papers  (2020-12-24T22:46:22Z)
• Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning [75.56839075060819]
  Traditional robotic approaches rely on an accurate model of the environment, a detailed description of how to perform the task, and a robust perception system to keep track of the current state. reinforcement learning approaches can operate directly from raw sensory inputs with only a reward signal to describe the task, but are extremely sample-inefficient and brittle. In this work, we combine the strengths of model-based methods with the flexibility of learning-based methods to obtain a general method that is able to overcome inaccuracies in the robotics perception/actuation pipeline.
  arXiv  Detail & Related papers  (2020-05-21T19:47:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.