Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic
Parsing
- URL: http://arxiv.org/abs/2010.03546v1
- Date: Wed, 7 Oct 2020 17:47:53 GMT
- Title: Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic
Parsing
- Authors: Xilun Chen, Asish Ghoshal, Yashar Mehdad, Luke Zettlemoyer and Sonal
Gupta
- Abstract summary: Task-oriented semantic parsing is a critical component of virtual assistants.
Recent advances in deep learning have enabled several approaches to successfully parse more complex queries.
We propose a novel method that outperforms a supervised neural model at a 10-fold data reduction.
- Score: 85.35582118010608
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Task-oriented semantic parsing is a critical component of virtual assistants,
which is responsible for understanding the user's intents (set reminder, play
music, etc.). Recent advances in deep learning have enabled several approaches
to successfully parse more complex queries (Gupta et al., 2018; Rongali et
al.,2020), but these models require a large amount of annotated training data
to parse queries on new domains (e.g. reminder, music).
In this paper, we focus on adapting task-oriented semantic parsers to
low-resource domains, and propose a novel method that outperforms a supervised
neural model at a 10-fold data reduction. In particular, we identify two
fundamental factors for low-resource domain adaptation: better representation
learning and better training techniques. Our representation learning uses BART
(Lewis et al., 2019) to initialize our model which outperforms encoder-only
pre-trained representations used in previous work. Furthermore, we train with
optimization-based meta-learning (Finn et al., 2017) to improve generalization
to low-resource domains. This approach significantly outperforms all baseline
methods in the experiments on a newly collected multi-domain task-oriented
semantic parsing dataset (TOPv2), which we release to the public.
Related papers
- Low-Resource Compositional Semantic Parsing with Concept Pretraining [35.35201295013346]
We present an architecture to perform such domain adaptation automatically.
We use a base seq2seq (sequence-to-sequence) architecture and augment it with a concept encoder that encodes intent and slot tags from the new domain.
We report few-shot and zero-shot results for compositional semantic parsing on the TOPv2 dataset.
arXiv Detail & Related papers (2023-01-24T04:27:27Z) - Combining Data Generation and Active Learning for Low-Resource Question Answering [23.755283239897132]
We propose a novel approach that combines data augmentation via question-answer generation with Active Learning to improve performance in low-resource settings.
Our findings show that our novel approach, where humans are incorporated in a data generation approach, boosts performance in the low-resource, domain-specific setting.
arXiv Detail & Related papers (2022-11-27T16:31:33Z) - On the Use of External Data for Spoken Named Entity Recognition [40.93448412171246]
Recent advances in self-supervised speech representations have made it feasible to consider learning models with limited labeled data.
We draw on a variety of approaches, including self-training, knowledge distillation, and transfer learning, and consider their applicability to both end-to-end models and pipeline approaches.
arXiv Detail & Related papers (2021-12-14T18:49:26Z) - Learning to Generalize Unseen Domains via Memory-based Multi-Source
Meta-Learning for Person Re-Identification [59.326456778057384]
We propose the Memory-based Multi-Source Meta-Learning framework to train a generalizable model for unseen domains.
We also present a meta batch normalization layer (MetaBN) to diversify meta-test features.
Experiments demonstrate that our M$3$L can effectively enhance the generalization ability of the model for unseen domains.
arXiv Detail & Related papers (2020-12-01T11:38:16Z) - Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
We introduce a fast optimization-based meta-learning method for few-shot classification.
Our strategy enables important aspects of the base learner objective to be learned during meta-training.
We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
arXiv Detail & Related papers (2020-10-01T15:59:31Z) - Meta-learning for Few-shot Natural Language Processing: A Survey [10.396506243272158]
Few-shot natural language processing (NLP) refers to NLP tasks that are accompanied with merely a handful of labeled examples.
This paper focuses on NLP domain, especially few-shot applications.
We try to provide clearer definitions, progress summary and some common datasets of applying meta-learning to few-shot NLP.
arXiv Detail & Related papers (2020-07-19T06:36:41Z) - Relation-Guided Representation Learning [53.60351496449232]
We propose a new representation learning method that explicitly models and leverages sample relations.
Our framework well preserves the relations between samples.
By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
arXiv Detail & Related papers (2020-07-11T10:57:45Z) - Pre-training Text Representations as Meta Learning [113.3361289756749]
We introduce a learning algorithm which directly optimize model's ability to learn text representations for effective learning of downstream tasks.
We show that there is an intrinsic connection between multi-task pre-training and model-agnostic meta-learning with a sequence of meta-train steps.
arXiv Detail & Related papers (2020-04-12T09:05:47Z) - Selecting Relevant Features from a Multi-domain Representation for
Few-shot Classification [91.67977602992657]
We propose a new strategy based on feature selection, which is both simpler and more effective than previous feature adaptation approaches.
We show that a simple non-parametric classifier built on top of such features produces high accuracy and generalizes to domains never seen during training.
arXiv Detail & Related papers (2020-03-20T15:44:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.