PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulation

• URL: http://arxiv.org/abs/2606.08414v1
• Date: Sun, 07 Jun 2026 02:27:53 GMT
• Title: PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulation
• Abstract summary: Existing approaches impose safety either prematurely during training or reactively via external guardrails at test time.<n>We propose Physical safety Alignment for Constrained Trajectories (PACT), a self-evolving post-training framework that projects pretrained diffusion policies onto constraint-feasible regions.<n>On simulated and real-world embodied manipulation benchmarks, PACT significantly reduces safety violations by 31.0% on average while improving task success by 30.7%.
• Score: 38.43792325467372
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion policies have achieved remarkable success in robotic manipulation, yet they often fail to satisfy strict physical constraints required for safe deployment. Existing approaches impose safety either prematurely during training or reactively via external guardrails at test time, limiting policy expressivity and overall scalability. We propose Physical safety Alignment for Constrained Trajectories (PACT), a self-evolving post-training framework that projects pretrained diffusion policies onto constraint-feasible regions without accessing demonstration data or task rewards. PACT distills constraint gradients into the diffusion model through a reverse-KL objective with dense supervision across timesteps. It incorporates a curriculum that progressively tightens constraints while maintaining theoretically bounded policy shift and monotone improvement, mitigating the safety-performance trade-off from catastrophic forgetting. On simulated and real-world embodied manipulation benchmarks, PACT significantly reduces safety violations by 31.0% on average while improving task success by 30.7%.

Related papers

• ActSafeGuard: Differentiable and Training-Aligned Constraint Enforcement for Flow-Matching Policies [13.296422778046148]
  We introduce ActSafeGuard, a differentiable and training-aligned safeguard layer for flow-matching based policies.<n>ActSafeGuard integrates hard action feasibility into policy learning, not merely treating safety as an inference-time external component.
  arXiv  Detail & Related papers  (2026-09-10T15:21:47Z)
• VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring [60.53395558502203]
  Vision-Language Embodied Safety Agent (VLESA) monitors human activities from egocentric video.<n>VLESA addresses intent-dependent safety where identical actions can be safe or dangerous depending on context.<n>An intent-action prediction agent is proposed to jointly infer goals and predict future actions from video.
  arXiv  Detail & Related papers  (2026-06-02T17:42:17Z)
• On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignment [54.30690671490447]
  Existing safety-alignment signals are largely response-level or off-policy.<n>FATE transforms verifier-scored failures into repair supervision without expert demonstrations.<n>FATE reduces attack success rate by 33.5%, harmful compliance by 82.6%, and improves external trajectory-safety diagnosis by 6.5%.
  arXiv  Detail & Related papers  (2026-05-12T09:56:28Z)
• BarrierSteer: LLM Safety via Learning Barrier Steering [83.12893815611052]
  BarrierSteer is a novel framework that formalizes safety by embedding learned non-linear safety constraints directly into the model's latent representation space.<n>We show that BarrierSteer substantially reduces adversarial success rates, decreases unsafe generations, and outperforms existing methods.
  arXiv  Detail & Related papers  (2026-02-23T18:19:46Z)
• Constrained Policy Optimization via Sampling-Based Weight-Space Projection [3.736063711613611]
  Safety-critical learning requires policies that improve performance without leaving the safe operating regime.<n>We study constrained policy learning where model parameters must satisfy unknown, rollout-based safety constraints.<n>We propose SCPO, a sampling-based weight-space projection method that enforces safety directly in parameter space without requiring gradient access to the constraint functions.
  arXiv  Detail & Related papers  (2025-12-15T19:00:01Z)
• Constraint-Aware Reinforcement Learning via Adaptive Action Scaling [6.6638441348404855]
  We propose a modular cost-aware regulator that scales the agent's actions based on predicted constraint violations.<n>The regulator is trained to minimize constraint violations while avoiding suppression of actions.<n>Our approach integrates seamlessly with off-policy RL methods such as SAC and TD3, and achieves state-of-the-art return-to-cost ratios on Safety Gym locomotion tasks.
  arXiv  Detail & Related papers  (2025-10-13T14:59:28Z)
• Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model [23.93820548551533]
  We propose FISOR (FeasIbility-guided Safe Offline RL), which allows safety constraint adherence, reward, and offline policy learning. In FISOR, the translated optimal policy for the translated optimization problem can be derived in a special form of weighted behavior cloning. We show that FISOR is the only method that can guarantee safety satisfaction in all tasks, while achieving top returns in most tasks.
  arXiv  Detail & Related papers  (2024-01-19T14:05:09Z)
• Safe Reinforcement Learning via Confidence-Based Filters [78.39359694273575]
  We develop a control-theoretic approach for certifying state safety constraints for nominal policies learned via standard reinforcement learning techniques. We provide formal safety guarantees, and empirically demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2022-07-04T11:43:23Z)
• Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations [64.39401322671803]
  This paper explores the possibility of safe RL algorithms with zero training-time safety violations. We propose an algorithm, Co-trained Barrier Certificate for Safe RL (CRABS), which iteratively learns barrier certificates, dynamics models, and policies.
  arXiv  Detail & Related papers  (2021-08-04T04:59:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.