Balancing Multi-Domain Corpora Learning for Open-Domain Response Generation

• URL: http://arxiv.org/abs/2205.02570v1
• Date: Thu, 5 May 2022 11:10:54 GMT
• Title: Balancing Multi-Domain Corpora Learning for Open-Domain Response Generation
• Authors: Yujie Xing, Jinglun Cai, Nils Barlaug, Peng Liu, Jon Atle Gulla
• Abstract summary: Open-domain conversational systems are assumed to generate equally good responses on multiple domains. This paper explores methods of generating relevant responses for each of multiple multi-domain corpora.
• Score: 3.3242685629646256
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Open-domain conversational systems are assumed to generate equally good responses on multiple domains. Previous work achieved good performance on the single corpus, but training and evaluating on multiple corpora from different domains are less studied. This paper explores methods of generating relevant responses for each of multiple multi-domain corpora. We first examine interleaved learning which intermingles multiple corpora as the baseline. We then investigate two multi-domain learning methods, labeled learning and multi-task labeled learning, which encode each corpus through a unique corpus embedding. Furthermore, we propose Domain-specific Frequency (DF), a novel word-level importance weight that measures the relative importance of a word for a specific corpus compared to other corpora. Based on DF, we propose weighted learning, a method that integrates DF to the loss function. We also adopt DF as a new evaluation metric. Extensive experiments show that our methods gain significant improvements on both automatic and human evaluation. We share our code and data for reproducibility

Related papers

• A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm [7.919690718820747]
  We propose a unified framework, dubbed Unified Domain Incremental Learning (UDIL), for domain incremental learning with memory. Our UDIL **unifies** various existing methods, and our theoretical analysis shows that UDIL always achieves a tighter generalization error bound compared to these methods. Empirical results show that our UDIL outperforms the state-of-the-art domain incremental learning methods on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-10-18T18:30:07Z)
• Multi-Source (Pre-)Training for Cross-Domain Measurement, Unit and Context Extraction [15.177664715250046]
  We present a cross-domain approach for automated measurement and context extraction based on pre-trained language models. We construct a multi-source, multi-domain corpus and train an end-to-end extraction pipeline. Our results suggest that multi-source training leads to the best overall results, while single-source training yields the best results for the respective individual domain.
  arXiv  Detail & Related papers  (2023-08-05T20:33:39Z)
• A Curriculum Learning Approach for Multi-domain Text Classification Using Keyword weight Ranking [17.71297141482757]
  We propose to use a curriculum learning strategy based on keyword weight ranking to improve the performance of multi-domain text classification models. The experimental results on the Amazon review and FDU-MTL datasets show that our curriculum learning strategy effectively improves the performance of multi-domain text classification models.
  arXiv  Detail & Related papers  (2022-10-27T03:15:26Z)
• Multi-Domain Long-Tailed Learning by Augmenting Disentangled Representations [80.76164484820818]
  There is an inescapable long-tailed class-imbalance issue in many real-world classification problems. We study this multi-domain long-tailed learning problem and aim to produce a model that generalizes well across all classes and domains. Built upon a proposed selective balanced sampling strategy, TALLY achieves this by mixing the semantic representation of one example with the domain-associated nuisances of another.
  arXiv  Detail & Related papers  (2022-10-25T21:54:26Z)
• MultiMatch: Multi-task Learning for Semi-supervised Domain Generalization [55.06956781674986]
  We resort to solving the semi-supervised domain generalization task, where there are a few label information in each source domain. We propose MultiMatch, extending FixMatch to the multi-task learning framework, producing the high-quality pseudo-label for SSDG. A series of experiments validate the effectiveness of the proposed method, and it outperforms the existing semi-supervised methods and the SSDG method on several benchmark DG datasets.
  arXiv  Detail & Related papers  (2022-08-11T14:44:33Z)
• Efficient Hierarchical Domain Adaptation for Pretrained Language Models [77.02962815423658]
  Generative language models are trained on diverse, general domain corpora. We introduce a method to scale domain adaptation to many diverse domains using a computationally efficient adapter approach.
  arXiv  Detail & Related papers  (2021-12-16T11:09:29Z)
• DABS: A Domain-Agnostic Benchmark for Self-Supervised Learning [6.040682281295584]
  We present DABS: a Domain-Agnostic Benchmark for Self-supervised learning. An algorithm is evaluated on seven diverse domains: natural images, multichannel sensor data, English text, speech recordings, multilingual text, chest x-rays, and images with text descriptions. We also present e-Mix and ShED: two baseline domain-agnostic algorithms.
  arXiv  Detail & Related papers  (2021-11-23T18:22:14Z)
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation [71.77083272602525]
  UDA attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain. We propose a contrastive learning approach that adapts category-wise centroids across domains. We extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels.
  arXiv  Detail & Related papers  (2021-05-05T11:55:53Z)
• TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning [53.32740707197856]
  We present a new state-of-the-art unsupervised method based on pre-trained Transformers and Sequential Denoising Auto-Encoder (TSDAE) It can achieve up to 93.1% of the performance of in-domain supervised approaches.
  arXiv  Detail & Related papers  (2021-04-14T17:02:18Z)
• Universal Representation Learning from Multiple Domains for Few-shot Classification [41.821234589075445]
  We propose to learn a single set of universal deep representations by distilling knowledge of multiple separately trained networks. We show that the universal representations can be further refined for previously unseen domains by an efficient adaptation step.
  arXiv  Detail & Related papers  (2021-03-25T13:49:12Z)
• Curriculum CycleGAN for Textual Sentiment Domain Adaptation with Multiple Sources [68.31273535702256]
  We propose a novel instance-level MDA framework, named curriculum cycle-consistent generative adversarial network (C-CycleGAN) C-CycleGAN consists of three components: (1) pre-trained text encoder which encodes textual input from different domains into a continuous representation space, (2) intermediate domain generator with curriculum instance-level adaptation which bridges the gap across source and target domains, and (3) task classifier trained on the intermediate domain for final sentiment classification. We conduct extensive experiments on three benchmark datasets and achieve substantial gains over state-of-the-art DA approaches.
  arXiv  Detail & Related papers  (2020-11-17T14:50:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.