Learning of Robot Safety Policies via Adversarial Synthetic Scenarios

• URL: http://arxiv.org/abs/2606.05952v1
• Date: Thu, 04 Jun 2026 09:51:57 GMT
• Title: Learning of Robot Safety Policies via Adversarial Synthetic Scenarios
• Abstract summary: We propose a framework for hazard-informed learning of robot safety policies through synthetic scenarios.<n>We model scenario generation as an adversarial game between two agents: a Red Team that explores the space of potential failures by constructing hazardous situations, and a Blue Team that incrementally refines safety policies to prevent them.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this work, we propose an agentic gamification framework for hazard-informed learning of robot safety policies through synthetic scenarios. We model scenario generation as an adversarial game between two agents: a Red Team that explores the space of potential failures by constructing hazardous situations, and a Blue Team that incrementally refines safety policies to prevent them. This iterative process enables efficient discovery of high-risk edge cases that are unlikely to be captured through random simulation or manual enumeration. By combining classical risk modeling with adversarial scenario generation and modern learning paradigms, this work provides a scalable pathway for embedding safety into Physical AI systems operating in complex real-world environments. The paper describes ongoing work. The contribution is a problem formulation and a proposed solution architecture.

Related papers

• Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses [168.50301366360344]
  Embodied Artificial Intelligence (Embodied AI) integrates perception, cognition, planning, and interaction into agents that operate in safety-critical environments.<n>Unlike digital AI systems, embodied agents must act under uncertain sensing, incomplete knowledge, and dynamic human-robot interactions.<n>This survey provides a comprehensive review of safety research in embodied AI, examining attacks and defenses across the full embodied pipeline.
  arXiv  Detail & Related papers  (2026-03-28T13:21:44Z)
• Provably Safe Trajectory Generation for Manipulators Under Motion and Environmental Uncertainties [26.41382315967256]
  This article proposes a novel risk-bounded motion planning framework to address the unmet need for safe optimal motion planning.<n>Our approach integrates a rigid manipulator deep Koopman operator (RM-DeKO) model to predict the robot's state distribution under motion uncertainty.<n>We then introduce an efficient hierarchical verification method that combines parallel physics simulations with sum-of-squares (SOS) programming as a filter for fine-grained, formal certification of collision risk.
  arXiv  Detail & Related papers  (2026-03-10T01:48:48Z)
• SafeAgent: Safeguarding LLM Agents via an Automated Risk Simulator [77.86600052899156]
  Large Language Model (LLM)-based agents are increasingly deployed in real-world applications.<n>We propose AutoSafe, the first framework that systematically enhances agent safety through fully automated synthetic data generation.<n>We show that AutoSafe boosts safety scores by 45% on average and achieves a 28.91% improvement on real-world tasks.
  arXiv  Detail & Related papers  (2025-05-23T10:56:06Z)
• AgentSGEN: Multi-Agent LLM in the Loop for Semantic Collaboration and GENeration of Synthetic Data [3.3186271052113843]
  scarcity of data presents a major obstacle to training AI systems for safety-critical applications, such as construction safety.<n>We propose a novel multi-agent framework that employs an iterative, in-the-loop collaboration between two agents.<n> powered by LLM's capabilities to reasoning and common-sense knowledge, this collaborative design produces synthetic images tailored to safety-critical scenarios.
  arXiv  Detail & Related papers  (2025-05-07T22:43:33Z)
• From Mystery to Mastery: Failure Diagnosis for Improving Manipulation Policies [35.18739716338974]
  This paper introduces Robot Manipulation Diagnosis (RoboMD), a systematic framework designed to automatically identify critical failure modes (FMs)<n>Considering the vast space of potential FMs in a pre-trained manipulation policy, we leverage deep reinforcement learning (deep RL) to explore and uncover these FMs.<n>We demonstrate RoboMD's effectiveness in diagnosing unknown failures in unstructured environments.
  arXiv  Detail & Related papers  (2024-12-03T20:34:51Z)
• Don't Let Your Robot be Harmful: Responsible Robotic Manipulation via Safety-as-Policy [53.048430683355804]
  Unthinking execution of human instructions in robotic manipulation can lead to severe safety risks.<n>We present Safety-as-policy, which includes (i) a world model to automatically generate scenarios containing safety risks and conduct virtual interactions, and (ii) a mental model to infer consequences with reflections.<n>We show that Safety-as-policy can avoid risks and efficiently complete tasks in both synthetic dataset and real-world experiments.
  arXiv  Detail & Related papers  (2024-11-27T12:27:50Z)
• Hazards in Daily Life? Enabling Robots to Proactively Detect and Resolve Anomalies [26.79399508110069]
  We argue that household robots should proactively detect such hazards or anomalies within the home. We leverage foundational models instead of relying on manually labeled data to build simulated environments. We demonstrate that our generated environment outperforms others in terms of task description and scene diversity.
  arXiv  Detail & Related papers  (2024-10-16T19:29:14Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.<n>Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.<n>However, the deployment of these agents in physical environments presents significant safety challenges.<n>This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• Compromising Embodied Agents with Contextual Backdoor Attacks [69.71630408822767]
  Large language models (LLMs) have transformed the development of embodied intelligence. This paper uncovers a significant backdoor security threat within this process. By poisoning just a few contextual demonstrations, attackers can covertly compromise the contextual environment of a black-box LLM.
  arXiv  Detail & Related papers  (2024-08-06T01:20:12Z)
• Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis [63.532413807686524]
  This paper addresses the problem of maintaining safety during training in Reinforcement Learning (RL) We propose a new architecture that handles the trade-off between efficient progress and safety during exploration.
  arXiv  Detail & Related papers  (2023-12-18T16:09:43Z)
• Towards Risk Modeling for Collaborative AI [5.941104748966331]
  Collaborative AI systems aim at working together with humans in a shared space to achieve a common goal. This setting imposes potentially hazardous circumstances due to contacts that could harm human beings. We introduce a risk modeling approach tailored to Collaborative AI systems.
  arXiv  Detail & Related papers  (2021-03-12T18:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.