Federated Marginal Personalization for ASR Rescoring

• URL: http://arxiv.org/abs/2012.00898v1
• Date: Tue, 1 Dec 2020 23:54:41 GMT
• Title: Federated Marginal Personalization for ASR Rescoring
• Authors: Zhe Liu, Fuchun Peng
• Abstract summary: Federated marginal personalization (FMP) is a novel method for continuously updating personalized neural network language models (NNLMs) on private devices using federated learning (FL) FMP regularly estimates global and personalized marginal distributions of words, and adjusts the probabilities from NNLMs by an adaptation factor that is specific to each word. Experiments on two speech evaluation datasets show modest word error rate (WER) reductions.
• Score: 13.086007347727206
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce federated marginal personalization (FMP), a novel method for continuously updating personalized neural network language models (NNLMs) on private devices using federated learning (FL). Instead of fine-tuning the parameters of NNLMs on personal data, FMP regularly estimates global and personalized marginal distributions of words, and adjusts the probabilities from NNLMs by an adaptation factor that is specific to each word. Our presented approach can overcome the limitations of federated fine-tuning and efficiently learn personalized NNLMs on devices. We study the application of FMP on second-pass ASR rescoring tasks. Experiments on two speech evaluation datasets show modest word error rate (WER) reductions. We also demonstrate that FMP could offer reasonable privacy with only a negligible cost in speech recognition accuracy.

Related papers

• DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation [15.023077875990614]
  Federated learning (FL) allows clients to collaboratively train a global model without sharing their local data with a server. Differential privacy (DP) addresses such leakage by providing formal privacy guarantees, with mechanisms that add randomness to the clients' contributions. We propose an adaptation method that can be combined with differential privacy and call it DP-DyLoRA.
  arXiv  Detail & Related papers  (2024-05-10T10:10:37Z)
• Personalized LLM Response Generation with Parameterized Memory Injection [19.417549781029233]
  Large Language Models (LLMs) have exhibited remarkable proficiency in comprehending and generating natural language. personalized LLM response generation holds the potential to offer substantial benefits for individuals in critical areas such as medical.
  arXiv  Detail & Related papers  (2024-04-04T16:20:34Z)
• Lattice Rescoring Based on Large Ensemble of Complementary Neural Language Models [50.164379437671904]
  We investigate the effectiveness of using a large ensemble of advanced neural language models (NLMs) for lattice rescoring on automatic speech recognition hypotheses. In experiments using a lecture speech corpus, by combining the eight NLMs and using context carry-over, we obtained a 24.4% relative word error rate reduction from the ASR 1-best baseline.
  arXiv  Detail & Related papers  (2023-12-20T04:52:24Z)
• Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes [53.4856038354195]
  Pre-trained large language models (LLMs) need fine-tuning to improve their responsiveness to natural language instructions. FedKSeed employs zeroth-order optimization with a finite set of random seeds. It significantly reduces transmission requirements between the server and clients to just a few random seeds.
  arXiv  Detail & Related papers  (2023-12-11T13:03:21Z)
• ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning [95.64041188351393]
  This paper endeavors to solve both the challenges of limited resources and personalization. We propose a method named ZOOPFL that uses Zeroth-Order Optimization for Personalized Federated Learning. To reduce the computation costs and enhance personalization, we propose input surgery to incorporate an auto-encoder with low-dimensional and client-specific embeddings.
  arXiv  Detail & Related papers  (2023-10-08T12:26:13Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)
• An Experimental Study on Private Aggregation of Teacher Ensemble Learning for End-to-End Speech Recognition [51.232523987916636]
  Differential privacy (DP) is one data protection avenue to safeguard user information used for training deep models by imposing noisy distortion on privacy data. In this work, we extend PATE learning to work with dynamic patterns, namely speech, and perform one very first experimental study on ASR to avoid acoustic data leakage.
  arXiv  Detail & Related papers  (2022-10-11T16:55:54Z)
• Neural-FST Class Language Model for End-to-End Speech Recognition [30.670375747577694]
  We propose a Neural-FST Class Language Model (NFCLM) for end-to-end speech recognition. We show that NFCLM significantly outperforms NNLM by 15.8% relative in terms of Word Error Rate.
  arXiv  Detail & Related papers  (2022-01-28T00:20:57Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Private Language Model Adaptation for Speech Recognition [15.726921748859393]
  Speech model adaptation is crucial to handle the discrepancy between server-side proxy training data and actual data received on users' local devices. We introduce an efficient approach on continuously adapting neural network language models (NNLMs) on private devices with applications on automatic speech recognition.
  arXiv  Detail & Related papers  (2021-09-28T00:15:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.