Resource-Constrained Federated Learning with Heterogeneous Labels and Models

• URL: http://arxiv.org/abs/2011.03206v1
• Date: Fri, 6 Nov 2020 06:23:47 GMT
• Title: Resource-Constrained Federated Learning with Heterogeneous Labels and Models
• Authors: Gautham Krishna Gudur, Bala Shyamala Balaji, Satheesh K. Perepu
• Abstract summary: We propose a framework with simple $alpha$-weighted federated aggregation of scores which leverages overlapping information gain across labels. We also demonstrate the on-device capabilities of our proposed framework by experimenting with federated learning and inference across different iterations on a Raspberry Pi 2.
• Score: 1.4824891788575418
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Various IoT applications demand resource-constrained machine learning mechanisms for different applications such as pervasive healthcare, activity monitoring, speech recognition, real-time computer vision, etc. This necessitates us to leverage information from multiple devices with few communication overheads. Federated Learning proves to be an extremely viable option for distributed and collaborative machine learning. Particularly, on-device federated learning is an active area of research, however, there are a variety of challenges in addressing statistical (non-IID data) and model heterogeneities. In addition, in this paper we explore a new challenge of interest -- to handle label heterogeneities in federated learning. To this end, we propose a framework with simple $\alpha$-weighted federated aggregation of scores which leverages overlapping information gain across labels, while saving bandwidth costs in the process. Empirical evaluation on Animals-10 dataset (with 4 labels for effective elucidation of results) indicates an average deterministic accuracy increase of at least ~16.7%. We also demonstrate the on-device capabilities of our proposed framework by experimenting with federated learning and inference across different iterations on a Raspberry Pi 2, a single-board computing platform.

Related papers

• Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning [52.6706505729803]
  We introduce Federated Learning (FL) to collaboratively train a decentralized shared model of Intrusion Detection Systems (IDS) FLEKD enables a more flexible aggregation method than conventional model fusion techniques. Experiment results show that the proposed approach outperforms local training and traditional FL in terms of both speed and performance.
  arXiv  Detail & Related papers  (2024-01-22T14:16:37Z)
• Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey [3.219812767529503]
  Federated learning (FL) offers a privacy-preserving trade-off between communication overhead and model accuracy. We outline the challengesFL has to overcome to be widely applicable in real-world applications.
  arXiv  Detail & Related papers  (2023-07-18T12:05:36Z)
• A Survey of Learning on Small Data: Generalization, Optimization, and Challenge [101.27154181792567]
  Learning on small data that approximates the generalization ability of big data is one of the ultimate purposes of AI. This survey follows the active sampling theory under a PAC framework to analyze the generalization error and label complexity of learning on small data. Multiple data applications that may benefit from efficient small data representation are surveyed.
  arXiv  Detail & Related papers  (2022-07-29T02:34:19Z)
• Federated Self-Supervised Learning in Heterogeneous Settings: Limits of a Baseline Approach on HAR [0.5039813366558306]
  We show that standard lightweight autoencoder and standard Federated Averaging fail to learn a robust representation for Human Activity Recognition. These findings advocate for a more intensive research effort in Federated Self Supervised Learning.
  arXiv  Detail & Related papers  (2022-07-17T14:15:45Z)
• 2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets [89.84774119537087]
  We design two transfer learning challenges around diagnostics and Brain-Computer-Interfacing (BCI) Task 1 is centred on medical diagnostics, addressing automatic sleep stage annotation across subjects. Task 2 is centred on Brain-Computer Interfacing (BCI), addressing motor imagery decoding across both subjects and data sets.
  arXiv  Detail & Related papers  (2022-02-14T12:12:20Z)
• Non-IID data and Continual Learning processes in Federated Learning: A long road ahead [58.720142291102135]
  Federated Learning is a novel framework that allows multiple devices or institutions to train a machine learning model collaboratively while preserving their data private. In this work, we formally classify data statistical heterogeneity and review the most remarkable learning strategies that are able to face it. At the same time, we introduce approaches from other machine learning frameworks, such as Continual Learning, that also deal with data heterogeneity and could be easily adapted to the Federated Learning settings.
  arXiv  Detail & Related papers  (2021-11-26T09:57:11Z)
• DQRE-SCnet: A novel hybrid approach for selecting users in Federated Learning with Deep-Q-Reinforcement Learning based on Spectral Clustering [1.174402845822043]
  Machine learning models based on sensitive data in the real-world promise advances in areas ranging from medical screening to disease outbreaks, agriculture, industry, defense science, and more. In many applications, learning participant communication rounds benefit from collecting their own private data sets, teaching detailed machine learning models on the real data, and sharing the benefits of using these models. Due to existing privacy and security concerns, most people avoid sensitive data sharing for training. Without each user demonstrating their local data to a central server, Federated Learning allows various parties to train a machine learning algorithm on their shared data jointly.
  arXiv  Detail & Related papers  (2021-11-07T15:14:29Z)
• Federated Learning: A Signal Processing Perspective [144.63726413692876]
  Federated learning is an emerging machine learning paradigm for training models across multiple edge devices holding local datasets, without explicitly exchanging the data. This article provides a unified systematic framework for federated learning in a manner that encapsulates and highlights the main challenges that are natural to treat using signal processing tools.
  arXiv  Detail & Related papers  (2021-03-31T15:14:39Z)
• Diverse Complexity Measures for Dataset Curation in Self-driving [80.55417232642124]
  We propose a new data selection method that exploits a diverse set of criteria that quantize interestingness of traffic scenes. Our experiments show that the proposed curation pipeline is able to select datasets that lead to better generalization and higher performance.
  arXiv  Detail & Related papers  (2021-01-16T23:45:02Z)
• Federated Learning with Heterogeneous Labels and Models for Mobile Activity Monitoring [0.7106986689736827]
  On-device Federated Learning proves to be an effective approach for distributed and collaborative machine learning. We propose a framework for federated label-based aggregation, which leverages overlapping information gain across activities. Empirical evaluation with the Heterogeneity Human Activity Recognition (HHAR) dataset on Raspberry Pi 2 indicates an average deterministic accuracy increase of at least 11.01%.
  arXiv  Detail & Related papers  (2020-12-04T11:44:17Z)
• Ternary Compression for Communication-Efficient Federated Learning [17.97683428517896]
  Federated learning provides a potential solution to privacy-preserving and secure machine learning. We propose a ternary federated averaging protocol (T-FedAvg) to reduce the upstream and downstream communication of federated learning systems. Our results show that the proposed T-FedAvg is effective in reducing communication costs and can even achieve slightly better performance on non-IID data.
  arXiv  Detail & Related papers  (2020-03-07T11:55:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.