Exploiting Label Skewness for Spiking Neural Networks in Federated Learning

• URL: http://arxiv.org/abs/2412.17305v2
• Date: Sun, 19 Jan 2025 06:58:01 GMT
• Title: Exploiting Label Skewness for Spiking Neural Networks in Federated Learning
• Authors: Di Yu, Xin Du, Linshan Jiang, Huijing Zhang, Shunwen Bai, Shuiguang Deng,
• Abstract summary: FedLEC incorporates intra-client label weight calibration to balance the learning intensity across local labels and inter-client knowledge distillation to mitigate local SNN model bias. Compared to eight state-of-the-art FL algorithms, FedLEC achieves an average accuracy improvement of approximately 11.59% for the global SNN model.
• Score: 18.846475069353364
• License:
• Abstract: The energy efficiency of deep spiking neural networks (SNNs) aligns with the constraints of resource-limited edge devices, positioning SNNs as a promising foundation for intelligent applications leveraging the extensive data collected by these devices. To address data privacy concerns when deploying SNNs on edge devices, federated learning (FL) facilitates collaborative model training by leveraging data distributed across edge devices without transmitting local data to a central server. However, existing FL approaches struggle with label-skewed data across devices, which leads to drift in local SNN models and degrades the performance of the global SNN model. In this paper, we propose a novel framework called FedLEC, which incorporates intra-client label weight calibration to balance the learning intensity across local labels and inter-client knowledge distillation to mitigate local SNN model bias caused by label absence. Extensive experiments with three different structured SNNs across five datasets (i.e., three non-neuromorphic and two neuromorphic datasets) demonstrate the efficiency of FedLEC. Compared to eight state-of-the-art FL algorithms, FedLEC achieves an average accuracy improvement of approximately 11.59% for the global SNN model under various label skew distribution settings.

Related papers

• Heterogeneous Federated Learning with Convolutional and Spiking Neural Networks [17.210940028586588]
  Federated learning (FL) has emerged as a promising paradigm for training models on decentralized data. This work benchmarks FL systems containing both convoluntional neural networks (CNNs) and biologically more plausible spiking neural networks (SNNs) Experimental results demonstrate that the CNN-SNN fusion framework exhibits the best performance.
  arXiv  Detail & Related papers  (2024-06-14T03:05:05Z)
• MatchNAS: Optimizing Edge AI in Sparse-Label Data Contexts via Automating Deep Neural Network Porting for Mobile Deployment [54.77943671991863]
  MatchNAS is a novel scheme for porting Deep Neural Networks to mobile devices. We optimise a large network family using both labelled and unlabelled data. We then automatically search for tailored networks for different hardware platforms.
  arXiv  Detail & Related papers  (2024-02-21T04:43:12Z)
• High-performance deep spiking neural networks with 0.3 spikes per neuron [9.01407445068455]
  It is hard to train biologically-inspired spiking neural networks (SNNs) than artificial neural networks (ANNs) We show that training deep SNN models achieves the exact same performance as that of ANNs. Our SNN accomplishes high-performance classification with less than 0.3 spikes per neuron, lending itself for an energy-efficient implementation.
  arXiv  Detail & Related papers  (2023-06-14T21:01:35Z)
• Sequential three-way decisions with a single hidden layer feedforward neural network [5.943305068876161]
  Three-way decisions strategy has been employed to construct network topology in a single hidden layer feedforward neural network (SFNN) This paper proposes STWD with an SFNN (STWD-SFNN) to enhance the performance of networks on structured datasets.
  arXiv  Detail & Related papers  (2023-03-14T02:26:34Z)
• Communication Trade-offs in Federated Learning of Spiking Neural Networks [0.0]
  Spiking Neural Networks (SNNs) are biologically inspired alternatives to conventional Artificial Neural Networks (ANNs) We consider SNNs in a federated learning setting where a high-quality global model is created by aggregating multiple local models from the clients without sharing any data. We evaluate the performance of the SNNs using a subset of the Spiking Heidelberg digits (SHD) dataset.
  arXiv  Detail & Related papers  (2023-02-27T19:12:03Z)
• Training Spiking Neural Networks with Local Tandem Learning [96.32026780517097]
  Spiking neural networks (SNNs) are shown to be more biologically plausible and energy efficient than their predecessors. In this paper, we put forward a generalized learning rule, termed Local Tandem Learning (LTL) We demonstrate rapid network convergence within five training epochs on the CIFAR-10 dataset while having low computational complexity.
  arXiv  Detail & Related papers  (2022-10-10T10:05:00Z)
• Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection [55.028065567756066]
  Out-of-distribution (OOD) detection has recently received much attention from the machine learning community due to its importance in deploying machine learning models in real-world applications. In this paper we propose an uncertainty quantification approach by modelling the distribution of features. We incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble neural networks (BE-SNNs) and overcome the feature collapse problem. We show that BE-SNNs yield superior performance on several OOD benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, FashionM
  arXiv  Detail & Related papers  (2022-06-26T16:00:22Z)
• SlimFL: Federated Learning with Superposition Coding over Slimmable Neural Networks [56.68149211499535]
  Federated learning (FL) is a key enabler for efficient communication and computing leveraging devices' distributed computing capabilities. This paper proposes a novel learning framework by integrating FL and width-adjustable slimmable neural networks (SNNs) We propose a communication and energy-efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models.
  arXiv  Detail & Related papers  (2022-03-26T15:06:13Z)
• Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks [52.93232352968347]
  This paper aims to integrate two synergetic technologies, federated learning (FL) and width-adjustable slimmable neural network (SNN) FL preserves data privacy by exchanging the locally trained models of mobile devices. SNNs are however non-trivial, particularly under wireless connections with time-varying channel conditions. We propose a communication and energy-efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models.
  arXiv  Detail & Related papers  (2021-12-05T11:17:17Z)
• Federated Learning with Spiking Neural Networks [13.09613811272936]
  Spiking Neural Networks (SNNs) are emerging as an energy-efficient alternative to the traditional Artificial Neural Networks (ANNs) We propose a federated learning framework for decentralized and privacy-preserving training of SNNs. We observe that SNNs outperform ANNs in terms of overall accuracy by over 15% when the data is distributed across a large number of clients in the federation.
  arXiv  Detail & Related papers  (2021-06-11T19:00:58Z)
• Learning to Solve the AC-OPF using Sensitivity-Informed Deep Neural Networks [52.32646357164739]
  We propose a deep neural network (DNN) to solve the solutions of the optimal power flow (ACOPF) The proposed SIDNN is compatible with a broad range of OPF schemes. It can be seamlessly integrated in other learning-to-OPF schemes.
  arXiv  Detail & Related papers  (2021-03-27T00:45:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.