Reinforcement Learning with Lie Group Orientations for Robotics

• URL: http://arxiv.org/abs/2409.11935v2
• Date: Tue, 5 Nov 2024 14:23:27 GMT
• Title: Reinforcement Learning with Lie Group Orientations for Robotics
• Authors: Martin Schuck, Jan Brüdigam, Sandra Hirche, Angela Schoellig,
• Abstract summary: We propose a simple modification of the network's input and output that adheres to the Lie group structure of orientations. As a result, we obtain an easy and efficient implementation that is directly usable with existing learning libraries. We briefly introduce Lie theory specifically for orientations in robotics to motivate and outline our approach.
• Score: 4.342261315851938
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Handling orientations of robots and objects is a crucial aspect of many applications. Yet, ever so often, there is a lack of mathematical correctness when dealing with orientations, especially in learning pipelines involving, for example, artificial neural networks. In this paper, we investigate reinforcement learning with orientations and propose a simple modification of the network's input and output that adheres to the Lie group structure of orientations. As a result, we obtain an easy and efficient implementation that is directly usable with existing learning libraries and achieves significantly better performance than other common orientation representations. We briefly introduce Lie theory specifically for orientations in robotics to motivate and outline our approach. Subsequently, a thorough empirical evaluation of different combinations of orientation representations for states and actions demonstrates the superior performance of our proposed approach in different scenarios, including: direct orientation control, end effector orientation control, and pick-and-place tasks.

Related papers

• A Primer on SO(3) Action Representations in Deep Reinforcement Learning [6.964881957695288]
  We show that representation-induced geometry strongly influences exploration and optimization.<n>Our results highlight that representing actions as tangent vectors in the local frame yields the most reliable results across algorithms.
  arXiv  Detail & Related papers  (2025-10-13T07:49:21Z)
• Learning and Optimization with 3D Orientations [39.146761527401424]
  This paper presents clearly, concisely and with unified notation all available representations, and "tricks" related to 3D orientations.<n>We benchmark them in representative scenarios.<n>We provide guidelines depending on the scenario, and make available a reference implementation of all the orientation math described.
  arXiv  Detail & Related papers  (2025-09-21T23:11:03Z)
• Search-Based Credit Assignment for Offline Preference-Based Reinforcement Learning [83.64755389431971]
  We introduce a Search-Based Preference Weighting scheme to unify two feedback sources.<n>For each transition in a preference labeled trajectory, SPW searches for the most similar state-action pairs from expert demonstrations.<n>These weights are then used to guide standard preference learning, enabling more accurate credit assignment.
  arXiv  Detail & Related papers  (2025-08-21T07:41:45Z)
• Feature-Based vs. GAN-Based Learning from Demonstrations: When and Why [50.191655141020505]
  This survey provides a comparative analysis of feature-based and GAN-based approaches to learning from demonstrations.<n>We argue that the dichotomy between feature-based and GAN-based methods is increasingly nuanced.
  arXiv  Detail & Related papers  (2025-07-08T11:45:51Z)
• AnyBody: A Benchmark Suite for Cross-Embodiment Manipulation [59.671764778486995]
  Generalizing control policies to novel embodiments remains a fundamental challenge in enabling scalable and transferable learning in robotics.<n>We introduce a benchmark for learning cross-embodiment manipulation, focusing on two foundational tasks-reach and push-across a diverse range of morphologies.<n>We evaluate the ability of different RL policies to learn from multiple morphologies and to generalize to novel ones.
  arXiv  Detail & Related papers  (2025-05-21T00:21:38Z)
• SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation [49.858348469657784]
  We introduce the concept of semantic orientation, which defines object orientations using natural language in a reference-frame-free manner. By integrating semantic orientation into a VLM system, we enable robots to generate manipulation actions with both positional and orientational constraints.
  arXiv  Detail & Related papers  (2025-02-18T18:59:02Z)
• Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization using Geometrical Information [68.10033984296247]
  This paper explores the domain of active localization, emphasizing the importance of viewpoint selection to enhance localization accuracy. Our contributions involve using a data-driven approach with a simple architecture designed for real-time operation, a self-supervised data training method, and the capability to consistently integrate our map into a planning framework tailored for real-world robotics applications.
  arXiv  Detail & Related papers  (2024-07-22T12:32:09Z)
• From Patches to Objects: Exploiting Spatial Reasoning for Better Visual Representations [2.363388546004777]
  We propose a novel auxiliary pretraining method that is based on spatial reasoning. Our proposed method takes advantage of a more flexible formulation of contrastive learning by introducing spatial reasoning as an auxiliary task for discriminative self-supervised methods.
  arXiv  Detail & Related papers  (2023-05-21T07:46:46Z)
• Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems [61.11799513362704]
  We propose learning an additional screening mechanism to identify discriminative clues commonly seen across instances and classes. We show that a common rationale detector can be learned by simply exploiting the GradCAM induced from the SSL objective.
  arXiv  Detail & Related papers  (2023-03-03T02:07:40Z)
• Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task [50.72283841720014]
  We propose a novel learning strategy that can improve reasoning about the effects of actions. We demonstrate the effectiveness of our proposed approach and discuss its advantages over previous baselines in terms of performance, data efficiency, and generalization capability.
  arXiv  Detail & Related papers  (2022-12-07T05:41:58Z)
• Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors [72.62423312645953]
  Humans intuitively solve tasks in versatile ways, varying their behavior in terms of trajectory-based planning and for individual steps. Current Imitation Learning algorithms often only consider unimodal expert demonstrations and act in a state-action-based setting. Instead, we combine a mixture of movement primitives with a distribution matching objective to learn versatile behaviors that match the expert's behavior and versatility.
  arXiv  Detail & Related papers  (2022-10-17T16:42:59Z)
• Dual Path Structural Contrastive Embeddings for Learning Novel Objects [6.979491536753043]
  Recent research shows that gaining information on a good feature space can be an effective solution to achieve favorable performance on few-shot tasks. We propose a simple but effective paradigm that decouples the tasks of learning feature representations and classifiers. Our method can still achieve promising results for both standard and generalized few-shot problems in either an inductive or transductive inference setting.
  arXiv  Detail & Related papers  (2021-12-23T04:43:31Z)
• Learning Oriented Remote Sensing Object Detection via Naive Geometric Computing [38.508709334835316]
  We propose a mechanism that learns the regression of horizontal proposals, oriented proposals, and rotation angles of objects in a consistent manner. Our proposed idea is simple and intuitive that can be readily implemented.
  arXiv  Detail & Related papers  (2021-12-01T13:58:42Z)
• CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning [138.40338621974954]
  CausalWorld is a benchmark for causal structure and transfer learning in a robotic manipulation environment. Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures.
  arXiv  Detail & Related papers  (2020-10-08T23:01:13Z)
• Orientation Attentive Robotic Grasp Synthesis with Augmented Grasp Map Representation [62.79160608266713]
  morphological characteristics in objects may offer a wide range of plausible grasping orientations that obfuscates the visual learning of robotic grasping. Existing grasp generation approaches are cursed to construct discontinuous grasp maps by aggregating annotations for drastically different orientations per grasping point. We propose a novel augmented grasp map representation, suitable for pixel-wise synthesis, that locally disentangles grasping orientations by partitioning the angle space into multiple bins.
  arXiv  Detail & Related papers  (2020-06-09T08:54:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.