Coordination-free Decentralised Federated Learning on Complex Networks: Overcoming Heterogeneity

• URL: http://arxiv.org/abs/2312.04504v1
• Date: Thu, 7 Dec 2023 18:24:19 GMT
• Title: Coordination-free Decentralised Federated Learning on Complex Networks: Overcoming Heterogeneity
• Authors: Lorenzo Valerio, Chiara Boldrini, Andrea Passarella, J\'anos Kert\'esz, M\'arton Karsai, Gerardo I\~niguez
• Abstract summary: Federated Learning (FL) is a framework for performing a learning task in an edge computing scenario. We propose a communication-efficient Decentralised Federated Learning (DFL) algorithm able to cope with them. Our solution allows devices communicating only with their direct neighbours to train an accurate model.
• Score: 2.6849848612544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Learning (FL) is a well-known framework for successfully performing a learning task in an edge computing scenario where the devices involved have limited resources and incomplete data representation. The basic assumption of FL is that the devices communicate directly or indirectly with a parameter server that centrally coordinates the whole process, overcoming several challenges associated with it. However, in highly pervasive edge scenarios, the presence of a central controller that oversees the process cannot always be guaranteed, and the interactions (i.e., the connectivity graph) between devices might not be predetermined, resulting in a complex network structure. Moreover, the heterogeneity of data and devices further complicates the learning process. This poses new challenges from a learning standpoint that we address by proposing a communication-efficient Decentralised Federated Learning (DFL) algorithm able to cope with them. Our solution allows devices communicating only with their direct neighbours to train an accurate model, overcoming the heterogeneity induced by data and different training histories. Our results show that the resulting local models generalise better than those trained with competing approaches, and do so in a more communication-efficient way.

Related papers

• Gradient-Congruity Guided Federated Sparse Training [31.793271982853188]
  Federated learning (FL) is a distributed machine learning technique that facilitates this process while preserving data privacy. FL also faces challenges such as high computational and communication costs regarding resource-constrained devices. We propose the Gradient-Congruity Guided Federated Sparse Training (FedSGC), a novel method that integrates dynamic sparse training and gradient congruity inspection into federated learning framework.
  arXiv  Detail & Related papers  (2024-05-02T11:29:48Z)
• Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey [3.219812767529503]
  Federated learning (FL) offers a privacy-preserving trade-off between communication overhead and model accuracy. We outline the challengesFL has to overcome to be widely applicable in real-world applications.
  arXiv  Detail & Related papers  (2023-07-18T12:05:36Z)
• Event-Triggered Decentralized Federated Learning over Resource-Constrained Edge Devices [12.513477328344255]
  Federated learning (FL) is a technique for distributed machine learning (ML) In traditional FL algorithms, trained models at the edge are periodically sent to a central server for aggregation. We develop a novel methodology for fully decentralized FL, where devices conduct model aggregation via cooperative consensus formation.
  arXiv  Detail & Related papers  (2022-11-23T00:04:05Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Federated Learning over Wireless IoT Networks with Optimized Communication and Resources [98.18365881575805]
  Federated learning (FL) as a paradigm of collaborative learning techniques has obtained increasing research attention. It is of interest to investigate fast responding and accurate FL schemes over wireless systems. We show that the proposed communication-efficient federated learning framework converges at a strong linear rate.
  arXiv  Detail & Related papers  (2021-10-22T13:25:57Z)
• Federated Learning: A Signal Processing Perspective [144.63726413692876]
  Federated learning is an emerging machine learning paradigm for training models across multiple edge devices holding local datasets, without explicitly exchanging the data. This article provides a unified systematic framework for federated learning in a manner that encapsulates and highlights the main challenges that are natural to treat using signal processing tools.
  arXiv  Detail & Related papers  (2021-03-31T15:14:39Z)
• Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networks [35.80470886180477]
  We propose a federated learning algorithm that is suitable for cellular wireless networks. We prove its convergence, and provide the optimal scheduling policy that maximizes the convergence rate.
  arXiv  Detail & Related papers  (2020-12-09T16:16:43Z)
• Wireless Communications for Collaborative Federated Learning [160.82696473996566]
  Internet of Things (IoT) devices may not be able to transmit their collected data to a central controller for training machine learning models. Google's seminal FL algorithm requires all devices to be directly connected with a central controller. This paper introduces a novel FL framework, called collaborative FL (CFL), which enables edge devices to implement FL with less reliance on a central controller.
  arXiv  Detail & Related papers  (2020-06-03T20:00:02Z)
• A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems [82.2513703281725]
  Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices. We present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems.
  arXiv  Detail & Related papers  (2020-03-18T05:56:27Z)
• Deep Learning for Ultra-Reliable and Low-Latency Communications in 6G Networks [84.2155885234293]
  We first summarize how to apply data-driven supervised deep learning and deep reinforcement learning in URLLC. To address these open problems, we develop a multi-level architecture that enables device intelligence, edge intelligence, and cloud intelligence for URLLC.
  arXiv  Detail & Related papers  (2020-02-22T14:38:11Z)
• Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks [8.456633924613456]
  Federated learning (FL) is emerging as a new paradigm to train machine learning models in distributed systems. The paper proposes a fully distributed (or server-less) learning approach: the proposed FL algorithms leverage the cooperation of devices that perform data operations inside the network. The approach lays the groundwork for integration of FL within 5G and beyond networks characterized by decentralized connectivity and computing.
  arXiv  Detail & Related papers  (2019-12-27T15:16:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.