SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking

• URL: http://arxiv.org/abs/2609.06061v1
• Date: Sat, 05 Sep 2026 12:43:41 GMT
• Title: SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking
• Abstract summary: We show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this feasibility collapse''<n>We empirically demonstrate that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance.
• Score: 0.4619828919345114
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: it avoids online optimization at inference, scales easily to many constraints via a single scalar per constraint, and is simpler to implement than commonly used Lagrangian methods. Instead of pricing violations through summed cost penalties, this approach makes violations structurally unprofitable by shortening the effective horizon for each violation. We identify a structural failure mode of this method class on terminal-navigation tasks: reaching a precise goal configuration while satisfying safety constraints that tighten along the final approach. When the goal sits inside the region close to where the constraints become active, the survival-weighted objective makes dwelling outside the goal region strictly preferable to entering, producing high constraint compliance with low task completion. We formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this ``feasibility collapse''. We empirically demonstrate that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance. Validation across two representative spacecraft platforms: a 6U-CubeSat spanning the mass and degree-of-freedom envelope of operational proximity operations, and a floating platform testbed for zero-shot sim-to-real transfer in our laboratory, supports the generality of these findings.

Related papers

• Directional Constraints for Efficient Exploration in Safe Reinforcement Learning [19.03722256244529]
  We propose an extension of the ATACOM framework to enforce constraints derived from prior knowledge of the system or learned directly from data.<n>Our proposed method, named ATACOM Directional Constraints (ATACOM-DC), significantly improves the safety-performance trade-off.<n>We evaluate our method across a range of challenging robotic control tasks in simulation, analyzing both constraint-violation costs and achieved task performance.
  arXiv  Detail & Related papers  (2026-07-14T13:59:20Z)
• Occluding the Solution Space: Planner-Agnostic Adversarial Attacks on Tolerance-Aware Manipulation [57.9825059129472]
  We propose a planner-agnostic attack framework for tolerance-aware manipulation.<n>Our approach shifts the evaluation paradigm to task-level feasibility over goal regions.<n>Online, we formulate the attack as a budgeted maximum-coverage optimization.
  arXiv  Detail & Related papers  (2026-07-04T08:14:44Z)
• Kilohertz-Safe: A Scalable Framework for Constrained Dexterous Retargeting [26.496984289625022]
  The paper proposes a scalable motion framework that reformulates the nonlinear problem into a convex program in joint differential space.<n>The proposed framework is validated through simulations and hardware experiments on the Wuji Hand platform.<n>The framework achieves high-frequency operation with an average latency of 9.05 ms, while over 95% of retargeted frames satisfy the safety criteria.
  arXiv  Detail & Related papers  (2026-03-31T03:28:04Z)
• BarrierSteer: LLM Safety via Learning Barrier Steering [83.12893815611052]
  BarrierSteer is a novel framework that formalizes safety by embedding learned non-linear safety constraints directly into the model's latent representation space.<n>We show that BarrierSteer substantially reduces adversarial success rates, decreases unsafe generations, and outperforms existing methods.
  arXiv  Detail & Related papers  (2026-02-23T18:19:46Z)
• Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learning [52.03884701766989]
  offline reinforcement learning (RL) algorithms typically impose constraints on action selection.<n>We propose a new neighborhood constraint that restricts action selection in the Bellman target to the union of neighborhoods of dataset actions.<n>We develop a simple yet effective algorithm, Adaptive Neighborhood-constrained Q learning (ANQ), to perform Q learning with target actions satisfying this constraint.
  arXiv  Detail & Related papers  (2025-11-04T13:42:05Z)
• Boundary-to-Region Supervision for Offline Safe Reinforcement Learning [56.150983204962735]
  Boundary-to-Region (B2R) is a framework that enables asymmetric conditioning through cost signal realignment.<n>B2R redefines CTG as a boundary constraint under a fixed safety budget, unifying the cost distribution of all feasible trajectories.<n> Experimental results show that B2R satisfies safety constraints in 35 out of 38 safety-critical tasks.
  arXiv  Detail & Related papers  (2025-09-30T03:38:20Z)
• Uniformly Safe RL with Objective Suppression for Multi-Constraint Safety-Critical Applications [73.58451824894568]
  The widely adopted CMDP model constrains the risks in expectation, which makes room for dangerous behaviors in long-tail states. In safety-critical domains, such behaviors could lead to disastrous outcomes. We propose Objective Suppression, a novel method that adaptively suppresses the task reward maximizing objectives according to a safety critic.
  arXiv  Detail & Related papers  (2024-02-23T23:22:06Z)
• Handling Long and Richly Constrained Tasks through Constrained Hierarchical Reinforcement Learning [20.280636126917614]
  Safety in goal directed Reinforcement Learning (RL) settings has typically been handled through constraints over trajectories. We propose a (safety) Constrained Search with Hierarchical Reinforcement Learning (CoSHRL) mechanism that combines an upper level constrained search agent with a low-level goal conditioned RL agent. A major advantage of CoSHRL is that it can handle constraints on the cost value distribution and can adjust to flexible constraint thresholds without retraining.
  arXiv  Detail & Related papers  (2023-02-21T12:57:12Z)
• Log Barriers for Safe Black-box Optimization with Application to Safe Reinforcement Learning [72.97229770329214]
  We introduce a general approach for seeking high dimensional non-linear optimization problems in which maintaining safety during learning is crucial. Our approach called LBSGD is based on applying a logarithmic barrier approximation with a carefully chosen step size. We demonstrate the effectiveness of our approach on minimizing violation in policy tasks in safe reinforcement learning.
  arXiv  Detail & Related papers  (2022-07-21T11:14:47Z)
• Adjacency constraint for efficient hierarchical reinforcement learning [25.15808501708926]
  Goal-conditioned Reinforcement Learning (HRL) is a promising approach for scaling up reinforcement learning (RL) techniques. HRL often suffers from training inefficiency as the action space of the high-level, i.e., the goal space, is large. We show that this problem can be effectively alleviated by restricting the high-level action space to a $k$-step adjacent region of the current state.
  arXiv  Detail & Related papers  (2021-10-30T09:26:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.