FedNST: Federated Noisy Student Training for Automatic Speech Recognition

• URL: http://arxiv.org/abs/2206.02797v1
• Date: Mon, 6 Jun 2022 16:18:45 GMT
• Title: FedNST: Federated Noisy Student Training for Automatic Speech Recognition
• Authors: Haaris Mehmood, Agnieszka Dobrowolska, Karthikeyan Saravanan, Mete Ozay
• Abstract summary: Federated Learning (FL) enables training state-of-the-art Automatic Speech Recognition (ASR) models on user devices (clients) in distributed systems. Key challenge facing practical adoption of FL for ASR is obtaining ground-truth labels on the clients. A promising alternative is using semi-/self-supervised learning approaches to leverage unlabelled user data.
• Score: 8.277567852741242
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) enables training state-of-the-art Automatic Speech Recognition (ASR) models on user devices (clients) in distributed systems, hence preventing transmission of raw user data to a central server. A key challenge facing practical adoption of FL for ASR is obtaining ground-truth labels on the clients. Existing approaches rely on clients to manually transcribe their speech, which is impractical for obtaining large training corpora. A promising alternative is using semi-/self-supervised learning approaches to leverage unlabelled user data. To this end, we propose a new Federated ASR method called FedNST for noisy student training of distributed ASR models with private unlabelled user data. We explore various facets of FedNST , such as training models with different proportions of unlabelled and labelled data, and evaluate the proposed approach on 1173 simulated clients. Evaluating FedNST on LibriSpeech, where 960 hours of speech data is split equally into server (labelled) and client (unlabelled) data, showed a 22.5% relative word error rate reduction (WERR) over a supervised baseline trained only on server data.

Related papers

• (FL)$^2$: Overcoming Few Labels in Federated Semi-Supervised Learning [4.803231218533992]
  Federated Learning (FL) is a distributed machine learning framework that trains accurate global models while preserving clients' privacy-sensitive data. Most FL approaches assume that clients possess labeled data, which is often not the case in practice. We propose $(FL)2$, a robust training method for unlabeled clients using sharpness-aware consistency regularization.
  arXiv  Detail & Related papers  (2024-10-30T17:15:02Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model [32.094290282897894]
  Federated learning aims to train a collective model from physically isolated clients while safeguarding the privacy of users' data. We propose a novel lightweight unsupervised federated learning approach that leverages unlabeled data on each client to perform lightweight model training and communication. Our proposed method greatly enhances model performance in comparison to CLIP's zero-shot predictions and even outperforms supervised federated learning benchmark methods.
  arXiv  Detail & Related papers  (2024-04-17T03:42:48Z)
• Navigating Data Heterogeneity in Federated Learning A Semi-Supervised Federated Object Detection [3.7398615061365206]
  Federated Learning (FL) has emerged as a potent framework for training models across distributed data sources. It faces challenges with limited high-quality labels and non-IID client data, particularly in applications like autonomous driving. We present a pioneering SSFOD framework, designed for scenarios where labeled data reside only at the server while clients possess unlabeled data.
  arXiv  Detail & Related papers  (2023-10-26T01:40:28Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings [56.93025161787725]
  Federated learning (FL) is a distributed machine learning paradigm that coordinates clients to train a model collaboratively without sharing local data. We propose an attribute inference attack framework that infers sensitive attribute information of the clients from shared gradients or model parameters. We show that the attribute inference attack is achievable for SER systems trained using FL.
  arXiv  Detail & Related papers  (2021-12-26T16:50:42Z)
• Federated Self-Training for Semi-Supervised Audio Recognition [0.23633885460047763]
  In this work, we study the problem of semi-supervised learning of audio models via self-training. We propose FedSTAR to exploit large-scale on-device unlabeled data to improve the generalization of audio recognition models.
  arXiv  Detail & Related papers  (2021-07-14T17:40:10Z)
• Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Data [27.51123821674191]
  We propose FEDENHANCE, an unsupervised federated learning (FL) approach for speech enhancement and separation. We simulate a real-world scenario where each client only has access to a few noisy recordings from a limited and disjoint number of speakers. Our experiments show that our approach achieves competitive enhancement performance compared to IID training on a single device.
  arXiv  Detail & Related papers  (2021-05-11T00:47:18Z)
• Federated Learning of User Authentication Models [69.93965074814292]
  We propose Federated User Authentication (FedUA), a framework for privacy-preserving training of machine learning models. FedUA adopts federated learning framework to enable a group of users to jointly train a model without sharing the raw inputs. We show our method is privacy-preserving, scalable with number of users, and allows new users to be added to training without changing the output layer.
  arXiv  Detail & Related papers  (2020-07-09T08:04:38Z)
• Uncertainty-aware Self-training for Text Classification with Few Labels [54.13279574908808]
  We study self-training as one of the earliest semi-supervised learning approaches to reduce the annotation bottleneck. We propose an approach to improve self-training by incorporating uncertainty estimates of the underlying neural network. We show our methods leveraging only 20-30 labeled samples per class for each task for training and for validation can perform within 3% of fully supervised pre-trained language models.
  arXiv  Detail & Related papers  (2020-06-27T08:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.