A Primer on SO(3) Action Representations in Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2510.11103v1
• Date: Mon, 13 Oct 2025 07:49:21 GMT
• Title: A Primer on SO(3) Action Representations in Deep Reinforcement Learning
• Authors: Martin Schuck, Sherif Samy, Angela P. Schoellig,
• Abstract summary: We show that representation-induced geometry strongly influences exploration and optimization.<n>Our results highlight that representing actions as tangent vectors in the local frame yields the most reliable results across algorithms.
• Score: 6.964881957695288
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many robotic control tasks require policies to act on orientations, yet the geometry of SO(3) makes this nontrivial. Because SO(3) admits no global, smooth, minimal parameterization, common representations such as Euler angles, quaternions, rotation matrices, and Lie algebra coordinates introduce distinct constraints and failure modes. While these trade-offs are well studied for supervised learning, their implications for actions in reinforcement learning remain unclear. We systematically evaluate SO(3) action representations across three standard continuous control algorithms, PPO, SAC, and TD3, under dense and sparse rewards. We compare how representations shape exploration, interact with entropy regularization, and affect training stability through empirical studies and analyze the implications of different projections for obtaining valid rotations from Euclidean network outputs. Across a suite of robotics benchmarks, we quantify the practical impact of these choices and distill simple, implementation-ready guidelines for selecting and using rotation actions. Our results highlight that representation-induced geometry strongly influences exploration and optimization and show that representing actions as tangent vectors in the local frame yields the most reliable results across algorithms.

Related papers

• Rotation-Adaptive Point Cloud Domain Generalization via Intricate Orientation Learning [34.424450834358204]
  We propose an innovative rotation-adaptive domain generalization framework for 3D point cloud analysis.<n>Our approach aims to alleviate orientational shifts by leveraging intricate samples in an iterative learning process.<n>We employ an orientation-aware contrastive learning framework that incorporates an orientation consistency loss and a margin separation loss.
  arXiv  Detail & Related papers  (2025-02-04T11:46:32Z)
• Reinforcement Learning with Lie Group Orientations for Robotics [4.342261315851938]
  We propose a simple modification of the network's input and output that adheres to the Lie group structure of orientations. As a result, we obtain an easy and efficient implementation that is directly usable with existing learning libraries. We briefly introduce Lie theory specifically for orientations in robotics to motivate and outline our approach.
  arXiv  Detail & Related papers  (2024-09-18T12:50:28Z)
• Learning Unorthogonalized Matrices for Rotation Estimation [83.94986875750455]
  Estimating 3D rotations is a common procedure for 3D computer vision. One form of representation -- rotation matrices -- is popular due to its continuity. We propose unorthogonalized Pseudo' Rotation Matrices (PRoM)
  arXiv  Detail & Related papers  (2023-12-01T09:56:29Z)
• Triangular Contrastive Learning on Molecular Graphs [2.8331075191137463]
  Triangular Contrastive Learning (TriCL) is a universal framework for trimodal contrastive learning. Triangular Area Loss is a novel intermodal contrastive loss that learns the angular geometry of the embedding space. We show that Triangular Area Loss can address the line-collapsing problem by discriminating modalities by angle.
  arXiv  Detail & Related papers  (2022-05-26T11:34:08Z)
• Unsupervised Learning on 3D Point Clouds by Clustering and Contrasting [11.64827192421785]
  unsupervised representation learning is a promising direction to auto-extract features without human intervention. This paper proposes a general unsupervised approach, named textbfConClu, to perform the learning of point-wise and global features.
  arXiv  Detail & Related papers  (2022-02-05T12:54:17Z)
• Composable Learning with Sparse Kernel Representations [110.19179439773578]
  We present a reinforcement learning algorithm for learning sparse non-parametric controllers in a Reproducing Kernel Hilbert Space. We improve the sample complexity of this approach by imposing a structure of the state-action function through a normalized advantage function. We demonstrate the performance of this algorithm on learning obstacle-avoidance policies in multiple simulations of a robot equipped with a laser scanner while navigating in a 2D environment.
  arXiv  Detail & Related papers  (2021-03-26T13:58:23Z)
• Self-supervised Geometric Perception [96.89966337518854]
  Self-supervised geometric perception is a framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels. We show that SGP achieves state-of-the-art performance that is on-par or superior to the supervised oracles trained using ground-truth labels.
  arXiv  Detail & Related papers  (2021-03-04T15:34:43Z)
• Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes [86.2129580231191]
  Adjoint Rigid Transform (ART) Network is a neural module which can be integrated with a variety of 3D networks. ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks. We will release our code and pre-trained models for further research.
  arXiv  Detail & Related papers  (2021-02-01T20:58:45Z)
• An Analysis of SVD for Deep Rotation Estimation [63.97835949897361]
  We present a theoretical analysis that shows SVD is the natural choice for projecting onto the rotation group. Our analysis shows simply replacing existing representations with the SVD orthogonalization procedure obtains state of the art performance in many deep learning applications.
  arXiv  Detail & Related papers  (2020-06-25T17:58:28Z)
• Discrete Action On-Policy Learning with Action-Value Critic [72.20609919995086]
  Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension. We construct a critic to estimate action-value functions, apply it on correlated actions, and combine these critic estimated action values to control the variance of gradient estimation. These efforts result in a new discrete action on-policy RL algorithm that empirically outperforms related on-policy algorithms relying on variance control techniques.
  arXiv  Detail & Related papers  (2020-02-10T04:23:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.