Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling

• URL: http://arxiv.org/abs/2203.08810v1
• Date: Tue, 15 Mar 2022 21:50:43 GMT
• Title: Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling
• Authors: Tiantian Feng and Shrikanth Narayanan
• Abstract summary: Speech Emotion Recognition (SER) application is frequently associated with privacy concerns. Federated learning (FL) is a distributed machine learning algorithm that coordinates clients to train a model collaboratively without sharing local data. In this work, we propose a semi-supervised federated learning framework, Semi-FedSER, that utilizes both labeled and unlabeled data samples to address the challenge of limited data samples in FL.
• Score: 43.17379040854574
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speech Emotion Recognition (SER) application is frequently associated with privacy concerns as it often acquires and transmits speech data at the client-side to remote cloud platforms for further processing. These speech data can reveal not only speech content and affective information but the speaker's identity, demographic traits, and health status. Federated learning (FL) is a distributed machine learning algorithm that coordinates clients to train a model collaboratively without sharing local data. This algorithm shows enormous potential for SER applications as sharing raw speech or speech features from a user's device is vulnerable to privacy attacks. However, a major challenge in FL is limited availability of high-quality labeled data samples. In this work, we propose a semi-supervised federated learning framework, Semi-FedSER, that utilizes both labeled and unlabeled data samples to address the challenge of limited labeled data samples in FL. We show that our Semi-FedSER can generate desired SER performance even when the local label rate l=20 using two SER benchmark datasets: IEMOCAP and MSP-Improv.

Related papers

• Partial Federated Learning [26.357723187375665]
  Federated Learning (FL) is a popular algorithm to train machine learning models on user data constrained to edge devices. We propose a new algorithm called Partial Federated Learning (PartialFL), where a machine learning model is trained using data where a subset of data modalities can be made available to the server.
  arXiv  Detail & Related papers  (2024-03-03T21:04:36Z)
• Communication-Efficient Personalized Federated Learning for Speech-to-Text Tasks [66.78640306687227]
  To protect privacy and meet legal regulations, federated learning (FL) has gained significant attention for training speech-to-text (S2T) systems. The commonly used FL approach (i.e., textscFedAvg) in S2T tasks typically suffers from extensive communication overhead. We propose a personalized federated S2T framework that introduces textscFedLoRA, a lightweight LoRA module for client-side tuning and interaction with the server, and textscFedMem, a global model equipped with a $k$-near
  arXiv  Detail & Related papers  (2024-01-18T15:39:38Z)
• Federated Representation Learning for Automatic Speech Recognition [20.641076546330986]
  Federated Learning (FL) is a privacy-preserving paradigm, allowing edge devices to learn collaboratively without sharing data. We bring Self-supervised Learning (SSL) and FL together to learn representations for Automatic Speech Recognition respecting data privacy constraints. We show that the pre-trained ASR encoder in FL performs as well as a centrally pre-trained model and produces an improvement of 12-15% (WER) compared to no pre-training.
  arXiv  Detail & Related papers  (2023-08-03T20:08:23Z)
• FedNoisy: Federated Noisy Label Learning Benchmark [53.73816587601204]
  Federated learning has gained popularity for distributed learning without aggregating sensitive data from clients. The distributed and isolated nature of data isolation may be complicated by data quality, making it more vulnerable to noisy labels. We serve the first standardized benchmark that can help researchers fully explore potential federated noisy settings.
  arXiv  Detail & Related papers  (2023-06-20T16:18:14Z)
• Semi-Supervised Federated Learning for Keyword Spotting [15.044022869136262]
  Keywords Spotting (KWS) is a critical aspect of audio-based applications on mobile devices and virtual assistants. Recent developments in Federated Learning (FL) have significantly expanded the ability to train machine learning models.
  arXiv  Detail & Related papers  (2023-05-09T00:46:12Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• SLICER: Learning universal audio representations using low-resource self-supervised pre-training [53.06337011259031]
  We present a new Self-Supervised Learning approach to pre-train encoders on unlabeled audio data. Our primary aim is to learn audio representations that can generalize across a large variety of speech and non-speech tasks.
  arXiv  Detail & Related papers  (2022-11-02T23:45:33Z)
• Split Federated Learning on Micro-controllers: A Keyword Spotting Showcase [1.4794135558227681]
  Federated Learning is proposed as a private learning scheme, using which users can locally train the model without collecting users' raw data to servers. In this work, we implement a simply SFL framework on the Arduino board and verify its correctness on the Chinese digits audio dataset for keyword spotting application with over 90% accuracy. On the English digits audio dataset, our SFL implementation achieves 13.89% higher accuracy compared to a state-of-the-art FL implementation.
  arXiv  Detail & Related papers  (2022-10-04T23:42:45Z)
• Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings [56.93025161787725]
  Federated learning (FL) is a distributed machine learning paradigm that coordinates clients to train a model collaboratively without sharing local data. We propose an attribute inference attack framework that infers sensitive attribute information of the clients from shared gradients or model parameters. We show that the attribute inference attack is achievable for SER systems trained using FL.
  arXiv  Detail & Related papers  (2021-12-26T16:50:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.