${\rm E}(3)$-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2308.11842v3
• Date: Sat, 25 May 2024 20:31:15 GMT
• Title: ${\rm E}(3)$-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning
• Abstract summary: We focus on exploiting Euclidean symmetries inherent in certain cooperative multi-agent reinforcement learning problems. We design neural network architectures with symmetric constraints embedded as an inductive bias for multi-agent actor-critic methods.
• Score: 7.712824077083934
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identification and analysis of symmetrical patterns in the natural world have led to significant discoveries across various scientific fields, such as the formulation of gravitational laws in physics and advancements in the study of chemical structures. In this paper, we focus on exploiting Euclidean symmetries inherent in certain cooperative multi-agent reinforcement learning (MARL) problems and prevalent in many applications. We begin by formally characterizing a subclass of Markov games with a general notion of symmetries that admits the existence of symmetric optimal values and policies. Motivated by these properties, we design neural network architectures with symmetric constraints embedded as an inductive bias for multi-agent actor-critic methods. This inductive bias results in superior performance in various cooperative MARL benchmarks and impressive generalization capabilities such as zero-shot learning and transfer learning in unseen scenarios with repeated symmetric patterns. The code is available at: https://github.com/dchen48/E3AC.

Related papers

• Harnessing Data Asymmetry: Manifold Learning in the Finsler World [53.951528511476994]
  We propose a Finsler manifold learning pipeline that constructs asymmetric dissimilarities and embeds in a Finsler space.<n>This greatly broadens the applicability of existing asymmetric embedders beyond traditionally directed data to any data.<n>On controlled synthetic and large real datasets, we show that our asymmetric pipeline reveals valuable information lost in the traditional pipeline.
  arXiv  Detail & Related papers  (2026-03-12T00:16:24Z)
• Equivariant Evidential Deep Learning for Interatomic Potentials [55.6997213490859]
  Uncertainty quantification is critical for assessing the reliability of machine learning interatomic potentials in molecular dynamics simulations.<n>Existing UQ approaches for MLIPs are often limited by high computational cost or suboptimal performance.<n>We propose textitEquivariant Evidential Deep Learning for Interatomic Potentials ($texte2$IP), a backbone-agnostic framework that models atomic forces and their uncertainty jointly.
  arXiv  Detail & Related papers  (2026-02-11T02:00:25Z)
• Reinforcement Learning Using known Invariances [54.91261509214309]
  This paper develops a theoretical framework for incorporating known group symmetries into kernel-based reinforcement learning.<n>We show that symmetry-aware RL achieves significantly better performance than their standard kernel counterparts.
  arXiv  Detail & Related papers  (2025-11-05T13:56:14Z)
• Symmetry and Generalisation in Neural Approximations of Renormalisation Transformations [11.337632710839166]
  We evaluate the role of symmetry and network expressivity in the generalisation behaviour of neural networks.<n>We consider simple multilayer perceptrons (MLPs) and graph neural networks (GNNs)<n>Our results reveal a competition between symmetry constraints and expressivity, with overly complex models generalising poorly.
  arXiv  Detail & Related papers  (2025-10-18T17:29:23Z)
• Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning [7.722898209589864]
  Learning dynamics is essential for model-based control and Reinforcement Learning in engineering systems.<n>We propose the Latent Mixture of Symmetries (Latent MoS), an expressive model that captures a mixture of symmetry-governed latent factors from complex dynamical measurements.
  arXiv  Detail & Related papers  (2025-10-04T00:06:31Z)
• Symmetries-enhanced Multi-Agent Reinforcement Learning [25.383183391244373]
  Multi-agent reinforcement learning has emerged as a powerful framework for enabling agents to learn complex, coordinated behaviors. Recent advancements have sought to alleviate those issues by embedding intrinsic symmetries of the systems in the policy. This paper presents a novel framework for embedding extrinsic symmetries in multi-agent system dynamics.
  arXiv  Detail & Related papers  (2025-01-02T08:41:31Z)
• Optimal Equivariant Architectures from the Symmetries of Matrix-Element Likelihoods [0.0]
  Matrix-Element Method (MEM) has long been a cornerstone of data analysis in high-energy physics. geometric deep learning has enabled neural network architectures that incorporate known symmetries directly into their design. This paper presents a novel approach that combines MEM-inspired symmetry considerations with equivariant neural network design for particle physics analysis.
  arXiv  Detail & Related papers  (2024-10-24T08:56:37Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• A Unified Framework to Enforce, Discover, and Promote Symmetry in Machine Learning [5.1105250336911405]
  We provide a unifying theoretical and methodological framework for incorporating symmetry into machine learning models. We show that enforcing and discovering symmetry are linear-algebraic tasks that are dual with respect to the bilinear structure of the Lie derivative. We propose a novel way to promote symmetry by introducing a class of convex regularization functions based on the Lie derivative and nuclear norm relaxation.
  arXiv  Detail & Related papers  (2023-11-01T01:19:54Z)
• Multi-constrained Symmetric Nonnegative Latent Factor Analysis for Accurately Representing Large-scale Undirected Weighted Networks [2.1797442801107056]
  An Undirected Weighted Network (UWN) is frequently encountered in a big-data-related application. An analysis model should carefully consider its symmetric-topology for describing an UWN's intrinsic symmetry. This paper proposes a Multi-constrained Symmetric Nonnegative Latent-factor-analysis model with two-fold ideas.
  arXiv  Detail & Related papers  (2023-06-06T14:13:16Z)
• Adaptive Log-Euclidean Metrics for SPD Matrix Learning [73.12655932115881]
  We propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM) The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks.
  arXiv  Detail & Related papers  (2023-03-26T18:31:52Z)
• Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principles [55.41644538483948]
  We design a deep-learning algorithm for the discovery and identification of the continuous group of symmetries present in a labeled dataset. We use fully connected neural networks to model the transformations symmetry and the corresponding generators. Our study also opens the door for using a machine learning approach in the mathematical study of Lie groups and their properties.
  arXiv  Detail & Related papers  (2023-01-13T16:25:25Z)
• Representation Theory for Geometric Quantum Machine Learning [0.0]
  Recent advances in classical machine learning have shown that creating models with inductive biases encoding the symmetries of a problem can greatly improve performance. Geometric Quantum Machine Learning (GQML) will play a crucial role in developing problem-specific and quantum-aware models. We present an introduction to representation theory tools from the optics of quantum learning, driven by key examples involving discrete and continuous groups.
  arXiv  Detail & Related papers  (2022-10-14T17:25:36Z)
• The Dynamics of Riemannian Robbins-Monro Algorithms [101.29301565229265]
  We propose a family of Riemannian algorithms generalizing and extending the seminal approximation framework of Robbins and Monro. Compared to their Euclidean counterparts, Riemannian algorithms are much less understood due to lack of a global linear structure on the manifold. We provide a general template of almost sure convergence results that mirrors and extends the existing theory for Euclidean Robbins-Monro schemes.
  arXiv  Detail & Related papers  (2022-06-14T12:30:11Z)
• On the Importance of Asymmetry for Siamese Representation Learning [53.86929387179092]
  Siamese networks are conceptually symmetric with two parallel encoders. We study the importance of asymmetry by explicitly distinguishing the two encoders within the network. We find the improvements from asymmetric designs generalize well to longer training schedules, multiple other frameworks and newer backbones.
  arXiv  Detail & Related papers  (2022-04-01T17:57:24Z)
• Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation [93.52573037053449]
  H-MARL (Hallucinated Multi-Agent Reinforcement Learning) learns successful equilibrium policies after a few interactions with the environment. We demonstrate our approach experimentally on an autonomous driving simulation benchmark.
  arXiv  Detail & Related papers  (2022-03-14T17:24:03Z)
• Symmetry Group Equivariant Architectures for Physics [52.784926970374556]
  In the domain of machine learning, an awareness of symmetries has driven impressive performance breakthroughs. We argue that both the physics community and the broader machine learning community have much to understand.
  arXiv  Detail & Related papers  (2022-03-11T18:27:04Z)
• Incorporating Symmetry into Deep Dynamics Models for Improved Generalization [24.363954435050264]
  We propose to improve accuracy and generalization by incorporating symmetries into convolutional neural networks. Our models are theoretically and experimentally robust to distributional shift by symmetry group transformations. Compared with image or text applications, our work is a significant step towards applying equivariant neural networks to high-dimensional systems.
  arXiv  Detail & Related papers  (2020-02-08T01:28:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.