Heterogeneous Multi-Robot Reinforcement Learning
- URL: http://arxiv.org/abs/2301.07137v1
- Date: Tue, 17 Jan 2023 19:05:17 GMT
- Title: Heterogeneous Multi-Robot Reinforcement Learning
- Authors: Matteo Bettini, Ajay Shankar, Amanda Prorok
- Abstract summary: Heterogeneous Graph Neural Network Proximal Policy Optimization is a paradigm for training heterogeneous MARL policies.
We present a characterization of techniques that homogeneous models can leverage to emulate heterogeneous behavior.
- Score: 7.22614468437919
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Cooperative multi-robot tasks can benefit from heterogeneity in the robots'
physical and behavioral traits. In spite of this, traditional Multi-Agent
Reinforcement Learning (MARL) frameworks lack the ability to explicitly
accommodate policy heterogeneity, and typically constrain agents to share
neural network parameters. This enforced homogeneity limits application in
cases where the tasks benefit from heterogeneous behaviors. In this paper, we
crystallize the role of heterogeneity in MARL policies. Towards this end, we
introduce Heterogeneous Graph Neural Network Proximal Policy Optimization
(HetGPPO), a paradigm for training heterogeneous MARL policies that leverages a
Graph Neural Network for differentiable inter-agent communication. HetGPPO
allows communicating agents to learn heterogeneous behaviors while enabling
fully decentralized training in partially observable environments. We
complement this with a taxonomical overview that exposes more heterogeneity
classes than previously identified. To motivate the need for our model, we
present a characterization of techniques that homogeneous models can leverage
to emulate heterogeneous behavior, and show how this "apparent heterogeneity"
is brittle in real-world conditions. Through simulations and real-world
experiments, we show that: (i) when homogeneous methods fail due to strong
heterogeneous requirements, HetGPPO succeeds, and, (ii) when homogeneous
methods are able to learn apparently heterogeneous behaviors, HetGPPO achieves
higher resilience to both training and deployment noise.
Related papers
- Semantically Rich Local Dataset Generation for Explainable AI in Genomics [0.716879432974126]
Black box deep learning models trained on genomic sequences excel at predicting the outcomes of different gene regulatory mechanisms.
We propose using Genetic Programming to generate datasets by evolving perturbations in sequences that contribute to their semantic diversity.
arXiv Detail & Related papers (2024-07-03T10:31:30Z) - Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models [83.02797560769285]
Data-Free Meta-Learning (DFML) aims to derive knowledge from a collection of pre-trained models without accessing their original data.
Current methods often overlook the heterogeneity among pre-trained models, which leads to performance degradation due to task conflicts.
We propose Task Groupings Regularization, a novel approach that benefits from model heterogeneity by grouping and aligning conflicting tasks.
arXiv Detail & Related papers (2024-05-26T13:11:55Z) - Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaboration [5.326588461041464]
We propose a novel framework named scalable and Heterogeneous Proximal Policy Optimization (SHPPO)
Our approach is based on the state-of-the-art backbone PPO-based algorithm as SHPPO.
SHPPO exhibits superior performance over the baselines in classic MARL environments like Starcraft Multi-Agent Challenge (SMAC) and Google Research Football (GRF)
arXiv Detail & Related papers (2024-04-05T03:02:57Z) - Histopathology Whole Slide Image Analysis with Heterogeneous Graph
Representation Learning [78.49090351193269]
We propose a novel graph-based framework to leverage the inter-relationships among different types of nuclei for WSI analysis.
Specifically, we formulate the WSI as a heterogeneous graph with "nucleus-type" attribute to each node and a semantic attribute similarity to each edge.
Our framework outperforms the state-of-the-art methods with considerable margins on various tasks.
arXiv Detail & Related papers (2023-07-09T14:43:40Z) - Conditionally Invariant Representation Learning for Disentangling
Cellular Heterogeneity [25.488181126364186]
This paper presents a novel approach that leverages domain variability to learn representations that are conditionally invariant to unwanted variability or distractors.
We apply our method to grand biological challenges, such as data integration in single-cell genomics.
Specifically, the proposed approach helps to disentangle biological signals from data biases that are unrelated to the target task or the causal explanation of interest.
arXiv Detail & Related papers (2023-07-02T12:52:41Z) - System Neural Diversity: Measuring Behavioral Heterogeneity in
Multi-Agent Learning [7.22614468437919]
We introduce System Neural Diversity (SND), a measure of behavioral heterogeneity for multi-agent systems.
We show how SND constitutes an important diagnostic tool to analyze latent properties of behavioral heterogeneity.
arXiv Detail & Related papers (2023-05-03T13:58:13Z) - Differentiable Agent-based Epidemiology [71.81552021144589]
We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation.
GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
arXiv Detail & Related papers (2022-07-20T07:32:02Z) - Efficient Model-based Multi-agent Reinforcement Learning via Optimistic
Equilibrium Computation [93.52573037053449]
H-MARL (Hallucinated Multi-Agent Reinforcement Learning) learns successful equilibrium policies after a few interactions with the environment.
We demonstrate our approach experimentally on an autonomous driving simulation benchmark.
arXiv Detail & Related papers (2022-03-14T17:24:03Z) - Automatic Generation of Individual Fuzzy Cognitive Maps from
Longitudinal Data [0.0]
Fuzzy Cognitive Maps (FCMs) are computational models that represent how factors (nodes) change over discrete interactions.
In this paper, we use Genetic Algorithms to create one FCM for each agent, thus providing the means to automatically create a virtual population with heterogeneous behaviors.
arXiv Detail & Related papers (2022-02-14T22:11:58Z) - Cooperative Policy Learning with Pre-trained Heterogeneous Observation
Representations [51.8796674904734]
We propose a new cooperative learning framework with pre-trained heterogeneous observation representations.
We employ an encoder-decoder based graph attention to learn the intricate interactions and heterogeneous representations.
arXiv Detail & Related papers (2020-12-24T04:52:29Z) - A Multi-Semantic Metapath Model for Large Scale Heterogeneous Network
Representation Learning [52.83948119677194]
We propose a multi-semantic metapath (MSM) model for large scale heterogeneous representation learning.
Specifically, we generate multi-semantic metapath-based random walks to construct the heterogeneous neighborhood to handle the unbalanced distributions.
We conduct systematical evaluations for the proposed framework on two challenging datasets: Amazon and Alibaba.
arXiv Detail & Related papers (2020-07-19T22:50:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.