Constraint-Enhanced Reinforcement Learning Based on Dynamic Decoupled Spherical Radial Squashing

• URL: http://arxiv.org/abs/2605.04185v1
• Date: Tue, 05 May 2026 18:24:46 GMT
• Title: Constraint-Enhanced Reinforcement Learning Based on Dynamic Decoupled Spherical Radial Squashing
• Abstract summary: Hard limits on how fast each joint can move per control step are unavoidable.<n>Existing methods impose isotropic ball-shaped constraints, exponentially under-covering the true feasible set.<n>This paper proposes Dynamic Decoupled Spherical Radial Squashing (DD-SRad)<n>Experiments with Unitree H1 and G1 humanoid robots confirm end-to-end optimality parameterized directly from official joint specifications.
• Score: 1.2036262042590717
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When deploying reinforcement learning policies to physical robots, actuator rate constraints -- hard limits on how fast each joint can move per control step -- are unavoidable. These limits vary substantially across joints due to differences in motor inertia, power bandwidth, and transmission stiffness, creating pronounced heterogeneity that existing methods fail to handle geometrically: the per-joint feasible region forms a high-dimensional box in action-increment space, yet QP projection and spherical parameterization methods impose isotropic ball-shaped constraints, exponentially under-covering the true feasible set as heterogeneity grows. This paper proposes Dynamic Decoupled Spherical Radial Squashing (DD-SRad), which resolves this mismatch by computing a position-adaptive radius independently for each actuator, achieving tight alignment with the true per-joint feasible region. DD-SRad satisfies per-step hard constraints with probability~1, preserves well-conditioned gradients throughout training, and admits exact policy gradient backpropagation with zero runtime solver overhead. MuJoCo benchmark experiments demonstrate the highest task return at zero constraint violation -- matching the unconstrained upper bound -- with 30%--50% improvement in constraint-space coverage over spherical baselines. High-fidelity IsaacLab simulations with Unitree H1 and G1 humanoid robots confirm end-to-end optimality parameterized directly from official joint specifications, validating a systematic pathway from hardware datasheets to safe deployment.

Related papers

• Across the Loss Landscape with Progressive Growth [51.366966420881646]
  We show how grow-and-optimize strategies to train toward flatter regions by viewing growth as progressive relaxation.<n>We empirically validate these predictions in controlled toy basins and in a realistic ResNet/CI-100 setting.<n>Although progressive subspace growth reliably produces flatter solutions, curvature reductions do not universally translate into improved test performance, highlighting subtleties in the flatness-generalization connection.
  arXiv  Detail & Related papers  (2026-08-25T13:50:48Z)
• A Task-Space Receding Horizon Controller for Fast Collision Avoidance [0.08594140167290099]
  Real-time collision avoidance for robotic manipulators requires fast reactions to unexpected obstacle motion.<n>Full model predictive control can provide this foresight, but its online cost may grow quickly with horizon length, model fidelity, and the number of active geometric constraints.<n>We present a task-space receding-horizon controller that uses a short contact-consistent rollout to generate a terminal kinematic reference.
  arXiv  Detail & Related papers  (2026-07-17T08:13:12Z)
• Trajectory Optimization for Collision-Aware Redundant Robotic Multi-Axis Additive Manufacturing by Constrained Gradient Projection [14.036000612440603]
  This work proposes a computational framework for collision-aware trajectory optimization in redundant robotic MAAM.<n> Experiments show that our method maintains a mean nozzle-position error below 10m, reduces maximum joint jerk by up to $77.6%$, and eliminates all sampled collision and orientation violations.
  arXiv  Detail & Related papers  (2026-06-29T04:18:54Z)
• Adaptive Smooth Tchebycheff Attention for Multi-Objective Policy Optimization [55.338784037660474]
  Multi- curvature reinforcement learning in domains requires balancing complex, non-fragile interference trade-offs between conflicting objectives.<n>In this paper we introduce a novel based real-time based real-time optimization on a robotic visual -- a proxy for monitoring ecosystems.
  arXiv  Detail & Related papers  (2026-05-12T21:32:57Z)
• Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control [50.28263951510334]
  We propose a hybrid hierarchical control framework for reactive dexterous grasping.<n>The proposed framework explicitly decouples high-level spatial intent from low-level joint execution.<n>We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline.
  arXiv  Detail & Related papers  (2026-05-05T04:49:38Z)
• Physics-Informed Reinforcement Learning of Spatial Density Velocity Potentials for Map-Free Racing [73.88859384645264]
  Out-Of-Distribution (OOD) generalization to various racetracks utilizes Machine Learning (ML) to encode the mathematical relation between sensor data and vehicle actuation for end-to-end control.<n>This paper presents a DRL method that parameterizes nonlinear vehicle dynamics from the spectral distribution of depth measurements with a non-geometric, physics-informed reward, to infer vehicle time-optimal and overtaking racing controls with an Artificial Neural Network (ANN)<n>The policy outperforms human demonstrations by 12% in OOD tracks on proportionally scaled hardware, by maximizing the friction circle with tire dynamics that resemble an empirical Pacejka tire model.
  arXiv  Detail & Related papers  (2026-04-10T17:12:07Z)
• Coordinated Manipulation of Hybrid Deformable-Rigid Objects in Constrained Environments [3.915966095774856]
  This work presents a quasi-static optimization-based manipulation planner that employs a strain-based Cosserat rod model.<n>It exploits the compliance of deformable links to maneuver through constraints while achieving task-space objectives for the object that are unreachable with rigid tools.<n>The proposed algorithm is validated in simulation on various hDLO systems and experimentally on a three-link hDLO manipulated in a constrained environment using a dual-arm robotic system.
  arXiv  Detail & Related papers  (2026-03-13T12:34:49Z)
• Fast and Safe Trajectory Optimization for Mobile Manipulators With Neural Configuration Space Distance Field [34.07401865897874]
  Generalized Space Fields (GCDF) can be used to extend base-arm coupling to mobile manipulators.<n>GCDF encodes whole-body configuration in configuration in unbounded space.<n>We develop a high-performance optimization solver centered on GCDF-based reasoning.
  arXiv  Detail & Related papers  (2026-01-26T14:55:26Z)
• GLOBE: Accurate and Generalizable PDE Surrogates using Domain-Inspired Architectures and Equivariances [0.0]
  GLOBE represents solutions as superpositions of learnable Green's-function-like kernels evaluated from boundary faces to targets.<n>On AirFRANS, GLOBE achieves substantial accuracy improvements.<n>Results show that rigorous physics- and domain-inspired inductive biases can achieve large gains in accuracy, generalizability, and practicality.
  arXiv  Detail & Related papers  (2025-11-19T20:23:51Z)
• Boundary-to-Region Supervision for Offline Safe Reinforcement Learning [56.150983204962735]
  Boundary-to-Region (B2R) is a framework that enables asymmetric conditioning through cost signal realignment.<n>B2R redefines CTG as a boundary constraint under a fixed safety budget, unifying the cost distribution of all feasible trajectories.<n> Experimental results show that B2R satisfies safety constraints in 35 out of 38 safety-critical tasks.
  arXiv  Detail & Related papers  (2025-09-30T03:38:20Z)
• RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking [50.200035833530876]
  RobotDancing is a simple, scalable framework that predicts residual joint targets to explicitly correct dynamics discrepancies.<n>It can track multi-minute, high-energy behaviors (jumps, spins, cartwheels) and deploys zero-shot to hardware with high motion tracking quality.
  arXiv  Detail & Related papers  (2025-09-25T03:30:34Z)
• SAFE--MA--RRT: Multi-Agent Motion Planning with Data-Driven Safety Certificates [6.77934423529734]
  This paper proposes a fully data-driven motion-planning framework for homogeneous linear multi-agent systems.<n>Each agent independently learns its closed-loop behavior from experimental data.<n>A sampling-based planner constructs a tree of such waypoints, where transitions are allowed only when adjacent ellipsoids overlap.
  arXiv  Detail & Related papers  (2025-09-04T17:34:59Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations [79.71184760864507]
  This article introduces a new physics-based method for rigid point set alignment called Fast Gravitational Approach (FGA) In FGA, the source and target point sets are interpreted as rigid particle swarms with masses interacting in a globally multiply-linked manner while moving in a simulated gravitational force field. We show that the new method class has characteristics not found in previous alignment methods.
  arXiv  Detail & Related papers  (2020-09-28T15:05:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.