Communication-Efficient Distributed Asynchronous ADMM

• URL: http://arxiv.org/abs/2508.12233v1
• Date: Sun, 17 Aug 2025 04:22:36 GMT
• Title: Communication-Efficient Distributed Asynchronous ADMM
• Authors: Sagar Shrestha,
• Abstract summary: We propose introducing coarse quantization to the data to be exchanged in aynchronous ADMM.<n>We experimentally verify the convergence of the proposed method for several distributed learning tasks, including neural networks.
• Score: 3.335932527835653
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In distributed optimization and federated learning, asynchronous alternating direction method of multipliers (ADMM) serves as an attractive option for large-scale optimization, data privacy, straggler nodes and variety of objective functions. However, communication costs can become a major bottleneck when the nodes have limited communication budgets or when the data to be communicated is prohibitively large. In this work, we propose introducing coarse quantization to the data to be exchanged in aynchronous ADMM so as to reduce communication overhead for large-scale federated learning and distributed optimization applications. We experimentally verify the convergence of the proposed method for several distributed learning tasks, including neural networks.

Related papers

• Heterogeneous Multi-agent Collaboration in UAV-assisted Mobile Crowdsensing Networks [6.226837215382989]
  Unmanned aerial vehicles (UAVs)-assisted mobile crowdsensing (MCS) has emerged as a promising paradigm for data collection.<n>We tackle challenges such as spectrum scarcity, device computation, and user mobility issues that hinder efficient coordination of sensing, communication, and resource allocation.
  arXiv  Detail & Related papers  (2025-09-28T02:13:19Z)
• Embedded Federated Feature Selection with Dynamic Sparse Training: Balancing Accuracy-Cost Tradeoffs [1.749521391198341]
  We present textitDynamic Sparse Federated Feature Selection (DSFFS), the first innovative embedded FFS that is efficient in both communication and computation.<n>During training, input-layer neurons, their connections, and hidden-layer connections are dynamically pruned and regrown, eliminating uninformative features.<n>Several experiments are conducted on nine real-world datasets, including biology, image, speech, and text.
  arXiv  Detail & Related papers  (2025-04-07T16:33:05Z)
• Distributed Event-Based Learning via ADMM [11.461617927469316]
  We consider a global distributed learning problem, where agents minimize an objective function by exchanging information over a network.<n>Our approach has two distinct settings: (i) It substantially reduces communication by triggering communication only when necessary, and (ii) it is to the data-distribution among the different agents.
  arXiv  Detail & Related papers  (2024-05-17T08:30:28Z)
• FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data [54.81695390763957]
  Federated learning is an emerging distributed machine learning method. We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate. We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
  arXiv  Detail & Related papers  (2023-09-18T12:35:05Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity [4.2059108111562935]
  Federated Learning (FL) is an emerging framework for distributed processing of large data volumes by edge devices. In this paper, we introduce a new FLAD FedADMM based protocol. We show that FedADMM consistently outperforms all baseline methods in terms of communication efficiency.
  arXiv  Detail & Related papers  (2022-04-07T15:58:33Z)
• CosSGD: Nonlinear Quantization for Communication-efficient Federated Learning [62.65937719264881]
  Federated learning facilitates learning across clients without transferring local data on these clients to a central server. We propose a nonlinear quantization for compressed gradient descent, which can be easily utilized in federated learning. Our system significantly reduces the communication cost by up to three orders of magnitude, while maintaining convergence and accuracy of the training process.
  arXiv  Detail & Related papers  (2020-12-15T12:20:28Z)
• Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge Computing [113.52575069030192]
  Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones and vehicles. Due to the limitations of communication costs and security requirements, it is of paramount importance to extract information in a decentralized manner instead of aggregating data to a fusion center. We consider the problem of learning model parameters in a multi-agent system with data locally processed via distributed edge nodes. A class of mini-batch alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
  arXiv  Detail & Related papers  (2020-10-02T10:41:59Z)
• Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM [52.12831959365598]
  We propose a communication-efficiently decentralized machine learning framework that solves a consensus optimization problem defined over a network of inter-connected workers. The proposed algorithm, Censored and Quantized Generalized GADMM, leverages the worker grouping and decentralized learning ideas of Group Alternating Direction Method of Multipliers (GADMM) Numerical simulations corroborate that CQ-GGADMM exhibits higher communication efficiency in terms of the number of communication rounds and transmit energy consumption without compromising the accuracy and convergence speed.
  arXiv  Detail & Related papers  (2020-09-14T14:18:19Z)
• Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications [55.65768284748698]
  Machine learning (ML) is a promising enabler for the fifth generation (5G) communication systems and beyond. This article aims to provide a holistic overview of relevant communication and ML principles, and thereby present communication-efficient and distributed learning frameworks with selected use cases.
  arXiv  Detail & Related papers  (2020-08-06T12:37:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.