Federated Self-Supervised Learning in Heterogeneous Settings: Limits of a Baseline Approach on HAR

• URL: http://arxiv.org/abs/2207.08187v1
• Date: Sun, 17 Jul 2022 14:15:45 GMT
• Title: Federated Self-Supervised Learning in Heterogeneous Settings: Limits of a Baseline Approach on HAR
• Authors: Sannara Ek, Romain Rombourg, Fran\c{c}ois Portet, Philippe Lalanda
• Abstract summary: We show that standard lightweight autoencoder and standard Federated Averaging fail to learn a robust representation for Human Activity Recognition. These findings advocate for a more intensive research effort in Federated Self Supervised Learning.
• Score: 0.5039813366558306
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning is a new machine learning paradigm dealing with distributed model learning on independent devices. One of the many advantages of federated learning is that training data stay on devices (such as smartphones), and only learned models are shared with a centralized server. In the case of supervised learning, labeling is entrusted to the clients. However, acquiring such labels can be prohibitively expensive and error-prone for many tasks, such as human activity recognition. Hence, a wealth of data remains unlabelled and unexploited. Most existing federated learning approaches that focus mainly on supervised learning have mostly ignored this mass of unlabelled data. Furthermore, it is unclear whether standard federated Learning approaches are suited to self-supervised learning. The few studies that have dealt with the problem have limited themselves to the favorable situation of homogeneous datasets. This work lays the groundwork for a reference evaluation of federated Learning with Semi-Supervised Learning in a realistic setting. We show that standard lightweight autoencoder and standard Federated Averaging fail to learn a robust representation for Human Activity Recognition with several realistic heterogeneous datasets. These findings advocate for a more intensive research effort in Federated Self Supervised Learning to exploit the mass of heterogeneous unlabelled data present on mobile devices.

Related papers

• A Survey on Class Imbalance in Federated Learning [6.632451878730774]
  Federated learning allows multiple client devices in a network to jointly train a machine learning model without direct exposure of clients' data. It has been found that models trained with federated learning usually have worse performance than their counterparts trained in the standard centralized learning mode.
  arXiv  Detail & Related papers  (2023-03-21T08:34:23Z)
• Federated Learning and Meta Learning: Approaches, Applications, and Directions [94.68423258028285]
  In this tutorial, we present a comprehensive review of FL, meta learning, and federated meta learning (FedMeta) Unlike other tutorial papers, our objective is to explore how FL, meta learning, and FedMeta methodologies can be designed, optimized, and evolved, and their applications over wireless networks.
  arXiv  Detail & Related papers  (2022-10-24T10:59:29Z)
• Comparative assessment of federated and centralized machine learning [0.0]
  Federated Learning (FL) is a privacy preserving machine learning scheme, where training happens with data federated across devices. In this paper, we discuss the various factors that affect the federated learning training, because of the non-IID distributed nature of the data. We show that federated learning does have an advantage in cost when the model sizes to be trained are not reasonably large.
  arXiv  Detail & Related papers  (2022-02-03T11:20:47Z)
• Non-IID data and Continual Learning processes in Federated Learning: A long road ahead [58.720142291102135]
  Federated Learning is a novel framework that allows multiple devices or institutions to train a machine learning model collaboratively while preserving their data private. In this work, we formally classify data statistical heterogeneity and review the most remarkable learning strategies that are able to face it. At the same time, we introduce approaches from other machine learning frameworks, such as Continual Learning, that also deal with data heterogeneity and could be easily adapted to the Federated Learning settings.
  arXiv  Detail & Related papers  (2021-11-26T09:57:11Z)
• FedSEAL: Semi-Supervised Federated Learning with Self-Ensemble Learning and Negative Learning [7.771967424619346]
  Federated learning (FL) is a popular decentralized and privacy-preserving machine learning (FL) framework. In this paper, we propose a new FL algorithm, called FedSEAL, to solve this Semi-Supervised Federated Learning (SSFL) problem. Our algorithm utilizes self-ensemble learning and complementary negative learning to enhance both the accuracy and the efficiency of clients' unsupervised learning on unlabeled data.
  arXiv  Detail & Related papers  (2021-10-15T03:03:23Z)
• Self-supervised Learning is More Robust to Dataset Imbalance [65.84339596595383]
  We investigate self-supervised learning under dataset imbalance. Off-the-shelf self-supervised representations are already more robust to class imbalance than supervised representations. We devise a re-weighted regularization technique that consistently improves the SSL representation quality on imbalanced datasets.
  arXiv  Detail & Related papers  (2021-10-11T06:29:56Z)
• Federated Self-Training for Semi-Supervised Audio Recognition [0.23633885460047763]
  In this work, we study the problem of semi-supervised learning of audio models via self-training. We propose FedSTAR to exploit large-scale on-device unlabeled data to improve the generalization of audio recognition models.
  arXiv  Detail & Related papers  (2021-07-14T17:40:10Z)
• Semi-supervised Federated Learning for Activity Recognition [9.720890017788676]
  Training deep learning models on in-home IoT sensory data is commonly used to recognise human activities. Recently, federated learning systems that use edge devices as clients to support local human activity recognition have emerged. We propose an activity recognition system that uses semi-supervised federated learning.
  arXiv  Detail & Related papers  (2020-11-02T09:47:14Z)
• Fairness in Semi-supervised Learning: Unlabeled Data Help to Reduce Discrimination [53.3082498402884]
  A growing specter in the rise of machine learning is whether the decisions made by machine learning models are fair. We present a framework of fair semi-supervised learning in the pre-processing phase, including pseudo labeling to predict labels for unlabeled data. A theoretical decomposition analysis of bias, variance and noise highlights the different sources of discrimination and the impact they have on fairness in semi-supervised learning.
  arXiv  Detail & Related papers  (2020-09-25T05:48:56Z)
• Fairness Constraints in Semi-supervised Learning [56.48626493765908]
  We develop a framework for fair semi-supervised learning, which is formulated as an optimization problem. We theoretically analyze the source of discrimination in semi-supervised learning via bias, variance and noise decomposition. Our method is able to achieve fair semi-supervised learning, and reach a better trade-off between accuracy and fairness than fair supervised learning.
  arXiv  Detail & Related papers  (2020-09-14T04:25:59Z)
• Federated and continual learning for classification tasks in a society of devices [59.45414406974091]
  Light Federated and Continual Consensus (LFedCon2) is a new federated and continual architecture that uses light, traditional learners. Our method allows powerless devices (such as smartphones or robots) to learn in real time, locally, continuously, autonomously and from users. In order to test our proposal, we have applied it in a heterogeneous community of smartphone users to solve the problem of walking recognition.
  arXiv  Detail & Related papers  (2020-06-12T12:37:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.