Modular Domain Adaptation for Conformer-Based Streaming ASR

• URL: http://arxiv.org/abs/2305.13408v1
• Date: Mon, 22 May 2023 18:49:35 GMT
• Title: Modular Domain Adaptation for Conformer-Based Streaming ASR
• Authors: Qiujia Li, Bo Li, Dongseong Hwang, Tara N. Sainath, Pedro M. Mengibar
• Abstract summary: We propose a framework that enables a single model to process multidomain data while keeping all parameters domain-specific. On a streaming Conformer transducer trained only on video caption data, experimental results show that an MDA-based model can reach similar performance as the multidomain model on other domains.
• Score: 28.398302172150636
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speech data from different domains has distinct acoustic and linguistic characteristics. It is common to train a single multidomain model such as a Conformer transducer for speech recognition on a mixture of data from all domains. However, changing data in one domain or adding a new domain would require the multidomain model to be retrained. To this end, we propose a framework called modular domain adaptation (MDA) that enables a single model to process multidomain data while keeping all parameters domain-specific, i.e., each parameter is only trained by data from one domain. On a streaming Conformer transducer trained only on video caption data, experimental results show that an MDA-based model can reach similar performance as the multidomain model on other domains such as voice search and dictation by adding per-domain adapters and per-domain feed-forward networks in the Conformer encoder.

Related papers

• DAM: Domain-Aware Module for Multi-Domain Dataset Condensation [25.14130854166058]
  Multi-Domain dataset Condensation (MDDC) aims to condense data that generalizes across both single-domain and multi-domain settings.<n> Domain-Aware Module (DAM) embeds domain-related features into each synthetic image via learnable spatial masks.<n>DAM consistently improves in-domain, out-of-domain, and cross-architecture performance over baseline dataset condensation methods.
  arXiv  Detail & Related papers  (2025-05-28T14:13:38Z)
• Virtual Classification: Modulating Domain-Specific Knowledge for Multidomain Crowd Counting [67.38137379297717]
  Multidomain crowd counting aims to learn a general model for multiple diverse datasets. Deep networks prefer modeling distributions of the dominant domains instead of all domains, which is known as domain bias. We propose a Modulating Domain-specific Knowledge Network (MDKNet) to handle the domain bias issue in multidomain crowd counting.
  arXiv  Detail & Related papers  (2024-02-06T06:49:04Z)
• Compositional Semantic Mix for Domain Adaptation in Point Cloud Segmentation [65.78246406460305]
  compositional semantic mixing represents the first unsupervised domain adaptation technique for point cloud segmentation. We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world)
  arXiv  Detail & Related papers  (2023-08-28T14:43:36Z)
• ML-BPM: Multi-teacher Learning with Bidirectional Photometric Mixing for Open Compound Domain Adaptation in Semantic Segmentation [78.19743899703052]
  Open compound domain adaptation (OCDA) considers the target domain as the compound of multiple unknown homogeneous. We introduce a multi-teacher framework with bidirectional photometric mixing to adapt to every target subdomain. We conduct an adaptive distillation to learn a student model and apply consistency regularization to improve the student generalization.
  arXiv  Detail & Related papers  (2022-07-19T03:30:48Z)
• Dynamic Instance Domain Adaptation [109.53575039217094]
  Most studies on unsupervised domain adaptation assume that each domain's training samples come with domain labels. We develop a dynamic neural network with adaptive convolutional kernels to generate instance-adaptive residuals to adapt domain-agnostic deep features to each individual instance. Our model, dubbed DIDA-Net, achieves state-of-the-art performance on several commonly used single-source and multi-source UDA datasets.
  arXiv  Detail & Related papers  (2022-03-09T20:05:54Z)
• Domain Adaptation via Prompt Learning [39.97105851723885]
  Unsupervised domain adaption (UDA) aims to adapt models learned from a well-annotated source domain to a target domain. We introduce a novel prompt learning paradigm for UDA, named Domain Adaptation via Prompt Learning (DAPL)
  arXiv  Detail & Related papers  (2022-02-14T13:25:46Z)
• Efficient Hierarchical Domain Adaptation for Pretrained Language Models [77.02962815423658]
  Generative language models are trained on diverse, general domain corpora. We introduce a method to scale domain adaptation to many diverse domains using a computationally efficient adapter approach.
  arXiv  Detail & Related papers  (2021-12-16T11:09:29Z)
• Unsupervised Domain Adaptation with Adapter [34.22467238579088]
  This paper explores an adapter-based fine-tuning approach for unsupervised domain adaptation. Several trainable adapter modules are inserted in a PrLM, and the embedded generic knowledge is preserved by fixing the parameters of the original PrLM. Elaborated experiments on two benchmark datasets are carried out, and the results demonstrate that our approach is effective with different tasks, dataset sizes, and domain similarities.
  arXiv  Detail & Related papers  (2021-11-01T02:50:53Z)
• Dynamic Transfer for Multi-Source Domain Adaptation [82.54405157719641]
  We present dynamic transfer to address domain conflicts, where the model parameters are adapted to samples. It breaks down source domain barriers and turns multi-source domains into a single-source domain. Experimental results show that, without using domain labels, our dynamic transfer outperforms the state-of-the-art method by more than 3%.
  arXiv  Detail & Related papers  (2021-03-19T01:22:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.