Zeroth-order Asynchronous Learning with Bounded Delays with a Use-case
in Resource Allocation in Communication Networks
- URL: http://arxiv.org/abs/2311.04604v1
- Date: Wed, 8 Nov 2023 11:12:27 GMT
- Title: Zeroth-order Asynchronous Learning with Bounded Delays with a Use-case
in Resource Allocation in Communication Networks
- Authors: Pourya Behmandpoor, Marc Moonen, Panagiotis Patrinos
- Abstract summary: This paper focuses on a scenario where agents collaborate toward a unified mission while potentially having distinct tasks.
Within this context, the objective for the agents is to optimize their local parameters based on the aggregate of local reward functions.
This paper presents theoretical convergence analyses and establishes a convergence rate for the proposed approach.
- Score: 12.216015676346032
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Distributed optimization has experienced a significant surge in interest due
to its wide-ranging applications in distributed learning and adaptation. While
various scenarios, such as shared-memory, local-memory, and consensus-based
approaches, have been extensively studied in isolation, there remains a need
for further exploration of their interconnections. This paper specifically
concentrates on a scenario where agents collaborate toward a unified mission
while potentially having distinct tasks. Each agent's actions can potentially
impact other agents through interactions. Within this context, the objective
for the agents is to optimize their local parameters based on the aggregate of
local reward functions, where only local zeroth-order oracles are available.
Notably, the learning process is asynchronous, meaning that agents update and
query their zeroth-order oracles asynchronously while communicating with other
agents subject to bounded but possibly random communication delays. This paper
presents theoretical convergence analyses and establishes a convergence rate
for the proposed approach. Furthermore, it addresses the relevant issue of deep
learning-based resource allocation in communication networks and conducts
numerical experiments in which agents, acting as transmitters, collaboratively
train their individual (possibly unique) policies to maximize a common
performance metric.
Related papers
- Distributed Event-Based Learning via ADMM [11.461617927469316]
We consider a distributed learning problem, where agents minimize a global objective function by exchanging information over a network.
Our approach has two distinct features: (i) It substantially reduces communication by triggering communication only when necessary, and (ii) it is agnostic to the data-distribution among the different agents.
arXiv Detail & Related papers (2024-05-17T08:30:28Z) - Learning Multi-Agent Communication from Graph Modeling Perspective [62.13508281188895]
We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph.
Our proposed approach, CommFormer, efficiently optimize the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner.
arXiv Detail & Related papers (2024-05-14T12:40:25Z) - Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning [57.652899266553035]
Decentralized and lifelong-adaptive multi-agent collaborative learning aims to enhance collaboration among multiple agents without a central server.
We propose DeLAMA, a decentralized multi-agent lifelong collaborative learning algorithm with dynamic collaboration graphs.
arXiv Detail & Related papers (2024-03-11T09:21:11Z) - Communication-Efficient Zeroth-Order Distributed Online Optimization:
Algorithm, Theory, and Applications [9.045332526072828]
This paper focuses on a multi-agent zeroth-order online optimization problem in a federated learning setting for target tracking.
The proposed solution is further analyzed in terms of errors and errors in two relevant applications.
arXiv Detail & Related papers (2023-06-09T03:51:45Z) - Exact Subspace Diffusion for Decentralized Multitask Learning [17.592204922442832]
Distributed strategies for multitask learning induce relationships between agents in a more nuanced manner, and encourage collaboration without enforcing consensus.
We develop a generalization of the exact diffusion algorithm for subspace constrained multitask learning over networks, and derive an accurate expression for its mean-squared deviation.
We verify numerically the accuracy of the predicted performance expressions, as well as the improved performance of the proposed approach over alternatives based on approximate projections.
arXiv Detail & Related papers (2023-04-14T19:42:19Z) - Compressed Regression over Adaptive Networks [58.79251288443156]
We derive the performance achievable by a network of distributed agents that solve, adaptively and in the presence of communication constraints, a regression problem.
We devise an optimized allocation strategy where the parameters necessary for the optimization can be learned online by the agents.
arXiv Detail & Related papers (2023-04-07T13:41:08Z) - Policy Evaluation in Decentralized POMDPs with Belief Sharing [39.550233049869036]
We consider a cooperative policy evaluation task in which agents are not assumed to observe the environment state directly.
We propose a fully decentralized belief forming strategy that relies on individual updates and on localized interactions over a communication network.
arXiv Detail & Related papers (2023-02-08T15:54:15Z) - Distributed Adaptive Learning Under Communication Constraints [54.22472738551687]
This work examines adaptive distributed learning strategies designed to operate under communication constraints.
We consider a network of agents that must solve an online optimization problem from continual observation of streaming data.
arXiv Detail & Related papers (2021-12-03T19:23:48Z) - Finite-Time Consensus Learning for Decentralized Optimization with
Nonlinear Gossiping [77.53019031244908]
We present a novel decentralized learning framework based on nonlinear gossiping (NGO), that enjoys an appealing finite-time consensus property to achieve better synchronization.
Our analysis on how communication delay and randomized chats affect learning further enables the derivation of practical variants.
arXiv Detail & Related papers (2021-11-04T15:36:25Z) - Distributed Bayesian Online Learning for Cooperative Manipulation [9.582645137247667]
We propose a novel distributed learning framework for the exemplary task of cooperative manipulation using Bayesian principles.
Using only local state information each agent obtains an estimate of the object dynamics and grasp kinematics.
Each estimate of the object dynamics and grasp kinematics is accompanied by a measure of uncertainty, which allows to guarantee a bounded prediction error with high probability.
arXiv Detail & Related papers (2021-04-09T13:03:09Z) - Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML.
We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML.
Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
arXiv Detail & Related papers (2020-10-06T16:51:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.