FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning

• URL: http://arxiv.org/abs/2608.09208v1
• Date: Mon, 10 Aug 2026 07:27:21 GMT
• Title: FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning
• Abstract summary: Decentralized intelligence systems with heterogeneous devices increasingly rely on decentralized federated learning (DFL)<n>This work introduces FedA2L, a method that dynamically adjusts layer-wise LRs based on model divergence signals.<n>By leveraging local update intensity and network consensus constraints, FedA2L seamlessly integrates into existing DFL protocols without additional communication or coordination.
• Score: 2.252594484945365
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized federated learning (DFL). However, DFL suffers from convergence inefficiency under data heterogeneity due to the use of a uniform learning rate (LR) that ignores layer-specific optimization needs. Foundational layers are responsible for maintaining network consensus, while specialized layers adapt to local data characteristics, leading to conflicting gradients and degraded performance under non-IID conditions. To address this fundamental tension, this work introduces FedA2L, a method that dynamically adjusts layer-wise LRs based on model divergence signals. By leveraging local update intensity and network consensus constraints, FedA2L seamlessly integrates into existing DFL protocols without additional communication or coordination. Extensive evaluations across DFL algorithms, various model architectures, and datasets demonstrate that FedA2L achieves up to 4.94 times faster convergence than vanilla DFL and reduces communication rounds by up to 59% compared to scheduler-based baselines. Furthermore, FedA2L exhibits resilience to severe data heterogeneity, larger network sizes, and sparse topologies, reducing communication overhead and establishing it as a versatile optimization tool for resource-constrained or large-scale distributed learning in edge and IoT deployments. The code is released at https://github.com/nclabteam/FedA2L.

Related papers

• On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensing [7.060465634216585]
  Federated learning (FL) enables collaborative training of deep learning models across decentralized image archives without requiring data centralization.<n>The presence of training data heterogeneity across clients (known as non-IID data) can impede convergence.<n>It is crucial to select an appropriate VLM adaptation strategy that balances the generalization ability with the communication and computational constraints.
  arXiv  Detail & Related papers  (2026-08-05T12:57:13Z)
• Air-Plan: Query-Optimized Topology Selection for Over-the-Air Decentralized Federated Learning [0.39373541926236766]
  Over-the-air (OTA) aggregation exploits the superposition property of wireless multiple-access channels to aggregate model updates from multiple devices within a single transmission slot.<n>We present AIRPLAN, a query-optimized topology selection framework for Over-the-Air Decentralized Federated Learning (OTA-DFL)<n>We show that AIRPLAN matches the oracle-optimal topology in 91.4% of workloads while introducing less than 1.8% overhead.
  arXiv  Detail & Related papers  (2026-07-05T12:10:47Z)
• Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization [56.95448807869383]
  Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning.<n>Recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage.<n>We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD)
  arXiv  Detail & Related papers  (2026-04-01T14:32:15Z)
• Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration [56.074760766965085]
  PRISM achieves a dynamics-aware framework that arbitrates data based on its degree of cognitive conflict with the model's existing knowledge.<n>Our findings suggest that disentangling data based on internal optimization regimes is crucial for scalable and robust agent alignment.
  arXiv  Detail & Related papers  (2026-01-12T05:43:20Z)
• Lightweight Federated Learning over Wireless Edge Networks [83.4818741890634]
  Federated (FL) is an alternative at network edge, but an alternative in wireless networks.<n>We derive a closed-form expression FL convergence gap transmission power, model pruning error, and quantization.<n> LTFL outperforms state-the-art schemes in experiments on real-world datasets.
  arXiv  Detail & Related papers  (2025-07-13T09:14:17Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• Topology Learning for Heterogeneous Decentralized Federated Learning over Unreliable D2D Networks [25.672506041978615]
  Decentralized federated learning (DFL) has attracted significant interest in wireless device-to-device (D2D) networks. We conduct a theoretical convergence analysis for DFL and derive a convergence bound. We develop a novel Topology Learning method considering the Representation Discrepancy and Unreliable Links in DFL, named ToLRDUL.
  arXiv  Detail & Related papers  (2023-12-21T07:01:18Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• Adaptive Federated Pruning in Hierarchical Wireless Networks [69.6417645730093]
  Federated Learning (FL) is a privacy-preserving distributed learning framework where a server aggregates models updated by multiple devices without accessing their private datasets. In this paper, we introduce model pruning for HFL in wireless networks to reduce the neural network scale. We show that our proposed HFL with model pruning achieves similar learning accuracy compared with the HFL without model pruning and reduces about 50 percent communication cost.
  arXiv  Detail & Related papers  (2023-05-15T22:04:49Z)
• Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks [44.37047471448793]
  In this paper, we advocate the integration of edge computing paradigm and parallel split learning (PSL) We propose an innovative PSL framework, namely, efficient parallel split learning (EPSL) to accelerate model training. We show that the proposed EPSL framework significantly decreases the training latency needed to achieve a target accuracy.
  arXiv  Detail & Related papers  (2023-03-26T16:09:48Z)
• Delay-Aware Hierarchical Federated Learning [7.292078085289465]
  The paper introduces delay-aware hierarchical federated learning (DFL) to improve the efficiency of distributed machine learning (ML) model training. During global synchronization, the cloud server consolidates local models with an outdated global model using a convex control algorithm. Numerical evaluations show DFL's superior performance in terms of faster global model, reduced convergence resource, and evaluations against communication delays.
  arXiv  Detail & Related papers  (2023-03-22T09:23:29Z)
• Fed-LAMB: Layerwise and Dimensionwise Locally Adaptive Optimization Algorithm [24.42828071396353]
  In the emerging paradigm of federated learning (FL), large amount of clients, such as mobile devices, are used to train on their respective data. Due to the low bandwidth, decentralized optimization methods need to shift the computation burden from those clients to those servers. We present Fed-LAMB, a novel learning method based on a layerwise, deep neural networks.
  arXiv  Detail & Related papers  (2021-10-01T16:54:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.