Continual learning using lattice-free MMI for speech recognition

• URL: http://arxiv.org/abs/2110.07055v1
• Date: Wed, 13 Oct 2021 22:11:11 GMT
• Title: Continual learning using lattice-free MMI for speech recognition
• Abstract summary: Continual learning (CL) or domain expansion is a popular topic for automatic speech recognition (ASR) acoustic modeling. Regularization-based CL for neural network acoustic models trained with the lattice-free maximum mutual information (LF-MMI) criterion is proposed. We show that a sequence-level LWF can improve the best average word error rate across all domains by up to 9.4% relative compared with using regular LWF.
• Score: 6.802401545890963
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning (CL), or domain expansion, recently became a popular topic for automatic speech recognition (ASR) acoustic modeling because practical systems have to be updated frequently in order to work robustly on types of speech not observed during initial training. While sequential adaptation allows tuning a system to a new domain, it may result in performance degradation on the old domains due to catastrophic forgetting. In this work we explore regularization-based CL for neural network acoustic models trained with the lattice-free maximum mutual information (LF-MMI) criterion. We simulate domain expansion by incrementally adapting the acoustic model on different public datasets that include several accents and speaking styles. We investigate two well-known CL techniques, elastic weight consolidation (EWC) and learning without forgetting (LWF), which aim to reduce forgetting by preserving model weights or network outputs. We additionally introduce a sequence-level LWF regularization, which exploits posteriors from the denominator graph of LF-MMI to further reduce forgetting. Empirical results show that the proposed sequence-level LWF can improve the best average word error rate across all domains by up to 9.4% relative compared with using regular LWF.

Related papers

• Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactions [120.64235456332916]
  Speech recognition systems increasingly integrate large language models (LLMs) to leverage their semantic knowledge.<n>We refine warm-adapted LLM-based ASR models by leveraging their own pre-adaptation base LLMs.<n>We propose Hybrid Search, a targeted correction strategy motivated by two observations.
  arXiv  Detail & Related papers  (2026-08-31T15:46:48Z)
• SLE-FNO: Single-Layer Extensions for Task-Agnostic Continual Learning in Fourier Neural Operators [0.4369550829556577]
  New experimental conditions or simulation regimes may differ significantly, requiring extrapolation and model updates without re-access to prior data.<n>This creates a need for continual learning (CL) frameworks that can adapt to distribution shifts while preventing catastrophic forgetting.<n>We introduce a new architecture-based approach (SLE-FNO) combining a Single-Layer Extension (SLE) with the Fourier Neural Operator (FNO) to support efficient CL.
  arXiv  Detail & Related papers  (2026-03-20T18:30:38Z)
• Self-Supervised Learning via Flow-Guided Neural Operator on Time-Series Data [57.85958428020496]
  Flow-Guided Neural Operator (FGNO) is a novel framework combining operator learning with flow matching for SSL training.<n>FGNO learns mappings in functional spaces by using Short-Time Fourier Transform to unify different time resolutions.<n>Unlike prior generative SSL methods that use noisy inputs during inference, we propose using clean inputs for representation extraction while learning representations with noise.
  arXiv  Detail & Related papers  (2026-02-12T18:54:57Z)
• Online Domain-aware LLM Decoding for Continual Domain Evolution [7.146027549101716]
  Domain knowledge evolves continuously through new regulations, products, services, and interaction patterns.<n>Disregarding this phenomenon, commonly referred to as concept drift, can significantly diminish a model's predictive accuracy.<n>Online Domain-aware Decoding framework (ODD) performs probability-level fusion between a base LLM and a prefix-tree prior.
  arXiv  Detail & Related papers  (2026-02-08T19:13:20Z)
• EReLiFM: Evidential Reliability-Aware Residual Flow Meta-Learning for Open-Set Domain Generalization under Noisy Labels [85.78886153628663]
  Open-Set Domain Generalization aims to enable deep learning models to recognize unseen categories in new domains.<n>Label noise hinders open-set domain generalization by corrupting source-domain knowledge.<n>We propose Evidential Reliability-Aware Residual Flow Meta-Learning (EReLiFM) to bridge domain gaps.
  arXiv  Detail & Related papers  (2025-10-14T16:23:11Z)
• Diffusion-Classifier Synergy: Reward-Aligned Learning via Mutual Boosting Loop for FSCIL [19.094835780362775]
  Few-Shot Class-Incremental Learning (FSCIL) challenges models to sequentially learn new classes from minimal examples.<n>Current FSCIL methods often struggle with generalization due to their reliance on limited datasets.<n>This paper introduces Diffusion-Classifier Synergy (DCS), a novel framework that establishes a mutual boosting loop between diffusion model and FSCIL classifier.
  arXiv  Detail & Related papers  (2025-10-04T01:48:52Z)
• SCA-LLM: Spectral-Attentive Channel Prediction with Large Language Models in MIMO-OFDM [63.80082983068669]
  An adapter is designed to bridge the domain gap between the channel state information (CSI) data and large language models (LLMs)<n>We propose a spectral-attentive framework, named SCA-LLM, for channel prediction in multiple-input multiple-output frequency division multiplexing (MIMO-OFDM) systems.
  arXiv  Detail & Related papers  (2025-09-09T20:43:12Z)
• Towards Efficient General Feature Prediction in Masked Skeleton Modeling [59.46799426434277]
  We propose a novel General Feature Prediction framework (GFP) for efficient mask skeleton modeling.<n>Our key innovation is replacing conventional low-level reconstruction with high-level feature prediction that spans from local motion patterns to global semantic representations.
  arXiv  Detail & Related papers  (2025-09-03T18:05:02Z)
• Input layer regularization and automated regularization hyperparameter tuning for myelin water estimation using deep learning [1.9594393134885413]
  We propose a novel deep learning method which combines classical regularization with data augmentation for estimating myelin water fraction (MWF) in the brain via biexponential analysis. In particular, we study the biexponential model, one of the signal models used for MWF estimation.
  arXiv  Detail & Related papers  (2025-01-30T00:56:28Z)
• Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs [76.40876036912537]
  Large Language Models (LLMs) demonstrate strong few-shot adaptability without requiring fine-tuning. Current Visual Foundation Models (VFMs) require explicit fine-tuning with sufficient tuning data. We propose a framework, LoRA Recycle, that distills a meta-LoRA from diverse pre-tuned LoRAs with a meta-learning objective.
  arXiv  Detail & Related papers  (2024-12-03T07:25:30Z)
• Recursive Learning of Asymptotic Variational Objectives [49.69399307452126]
  General state-space models (SSMs) are widely used in statistical machine learning and are among the most classical generative models for sequential time-series data. Online sequential IWAE (OSIWAE) allows for online learning of both model parameters and a Markovian recognition model for inferring latent states. This approach is more theoretically well-founded than recently proposed online variational SMC methods.
  arXiv  Detail & Related papers  (2024-11-04T16:12:37Z)
• LLM-TS Integrator: Integrating LLM for Enhanced Time Series Modeling [5.853711797849859]
  Time series(TS) modeling is essential in dynamic systems like weather prediction and anomaly detection. Recent studies utilize Large Language Models (LLMs) for TS modeling, leveraging their powerful pattern recognition capabilities.
  arXiv  Detail & Related papers  (2024-10-21T20:29:46Z)
• Temporal-Difference Variational Continual Learning [89.32940051152782]
  A crucial capability of Machine Learning models in real-world applications is the ability to continuously learn new tasks. In Continual Learning settings, models often struggle to balance learning new tasks with retaining previous knowledge. We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning [17.614980614656407]
  We propose Continual Generative training for Incremental prompt-Learning. We exploit Variational Autoencoders to learn class-conditioned distributions. We show that such a generative replay approach can adapt to new tasks while improving zero-shot capabilities.
  arXiv  Detail & Related papers  (2024-07-22T16:51:28Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters [65.15700861265432]
  We present a parameter-efficient continual learning framework to alleviate long-term forgetting in incremental learning with vision-language models. Our approach involves the dynamic expansion of a pre-trained CLIP model, through the integration of Mixture-of-Experts (MoE) adapters. To preserve the zero-shot recognition capability of vision-language models, we introduce a Distribution Discriminative Auto-Selector.
  arXiv  Detail & Related papers  (2024-03-18T08:00:23Z)
• CLIPood: Generalizing CLIP to Out-of-Distributions [73.86353105017076]
  Contrastive language-image pre-training (CLIP) models have shown impressive zero-shot ability, but the further adaptation of CLIP on downstream tasks undesirably degrades OOD performances. We propose CLIPood, a fine-tuning method that can adapt CLIP models to OOD situations where both domain shifts and open classes may occur on unseen test data. Experiments on diverse datasets with different OOD scenarios show that CLIPood consistently outperforms existing generalization techniques.
  arXiv  Detail & Related papers  (2023-02-02T04:27:54Z)
• Generalized Variational Continual Learning [33.194866396158005]
  Two main approaches to continuous learning are Online Elastic Weight Consolidation and Variational Continual Learning. We show that applying this modification to mitigate Online EWC as a limiting case, allowing baselines between the two approaches. In order to the observed overpruning effect of VI, we take inspiration from a common multi-task architecture, mitigate neural networks with task-specific FiLM layers.
  arXiv  Detail & Related papers  (2020-11-24T19:07:39Z)
• Frequency-based Automated Modulation Classification in the Presence of Adversaries [17.930854969511046]
  We present a novel receiver architecture consisting of deep learning models capable of withstanding transferable adversarial interference. In this work, we demonstrate classification performance improvements greater than 30% on recurrent neural networks (RNNs) and greater than 50% on convolutional neural networks (CNNs)
  arXiv  Detail & Related papers  (2020-11-02T17:12:22Z)
• Early Stage LM Integration Using Local and Global Log-Linear Combination [46.91755970827846]
  Sequence-to-sequence models with an implicit alignment mechanism (e.g. attention) are closing the performance gap towards traditional hybrid hidden Markov models (HMM) One important factor to improve word error rate in both cases is the use of an external language model (LM) trained on large text-only corpora. We present a novel method for language model integration into implicit-alignment based sequence-to-sequence models.
  arXiv  Detail & Related papers  (2020-05-20T13:49:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.