Language Model is All You Need: Natural Language Understanding as Question Answering

• URL: http://arxiv.org/abs/2011.03023v1
• Date: Thu, 5 Nov 2020 18:31:22 GMT
• Title: Language Model is All You Need: Natural Language Understanding as Question Answering
• Authors: Mahdi Namazifar, Alexandros Papangelis, Gokhan Tur, Dilek Hakkani-T\"ur
• Abstract summary: We study the use of a specific family of transfer learning, where the target domain is mapped to the source domain. We show that our approach could reduce the amount of required data for the same performance by up to a factor of 10.
• Score: 75.26830242165742
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Different flavors of transfer learning have shown tremendous impact in advancing research and applications of machine learning. In this work we study the use of a specific family of transfer learning, where the target domain is mapped to the source domain. Specifically we map Natural Language Understanding (NLU) problems to QuestionAnswering (QA) problems and we show that in low data regimes this approach offers significant improvements compared to other approaches to NLU. Moreover we show that these gains could be increased through sequential transfer learning across NLU problems from different domains. We show that our approach could reduce the amount of required data for the same performance by up to a factor of 10.

Related papers

• Exploring Language Model Generalization in Low-Resource Extractive QA [57.14068405860034]
  We investigate Extractive Question Answering (EQA) with Large Language Models (LLMs) under domain drift. We devise a series of experiments to empirically explain the performance gap.
  arXiv  Detail & Related papers  (2024-09-27T05:06:43Z)
• Characterization of effects of transfer learning across domains and languages [0.0]
  Transfer learning (TL) from pre-trained neural language models has emerged as a powerful technique over the years. We investigate how TL affects the performance of popular pre-trained models over three natural language processing (NLP) tasks.
  arXiv  Detail & Related papers  (2022-10-03T17:17:07Z)
• Deep invariant networks with differentiable augmentation layers [87.22033101185201]
  Methods for learning data augmentation policies require held-out data and are based on bilevel optimization problems. We show that our approach is easier and faster to train than modern automatic data augmentation techniques.
  arXiv  Detail & Related papers  (2022-02-04T14:12:31Z)
• A Bayesian Approach to (Online) Transfer Learning: Theory and Algorithms [6.193838300896449]
  We study transfer learning from a Bayesian perspective, where a parametric statistical model is used. Specifically, we study three variants of transfer learning problems, instantaneous, online, and time-variant transfer learning. For each problem, we define an appropriate objective function, and provide either exact expressions or upper bounds on the learning performance. Examples show that the derived bounds are accurate even for small sample sizes.
  arXiv  Detail & Related papers  (2021-09-03T08:43:29Z)
• Transfer Learning for Aided Target Recognition: Comparing Deep Learning to other Machine Learning Approaches [0.0]
  Aided target recognition (AiTR) is an important problem with applications across industry and defense. Deep learning (DL) provides exceptional modeling flexibility and accuracy on recent real world problems. Our goal is to address this shortcoming by comparing transfer learning within a DL framework to other ML approaches across transfer tasks and datasets.
  arXiv  Detail & Related papers  (2020-11-25T14:25:49Z)
• What is being transferred in transfer learning? [51.6991244438545]
  We show that when training from pre-trained weights, the model stays in the same basin in the loss landscape. We present that when training from pre-trained weights, the model stays in the same basin in the loss landscape and different instances of such model are similar in feature space and close in parameter space.
  arXiv  Detail & Related papers  (2020-08-26T17:23:40Z)
• Limits of Transfer Learning [0.0]
  We show the need to carefully select which sets of information to transfer and the need for dependence between transferred information and target problems. These results build on the algorithmic search framework for machine learning, allowing the results to apply to a wide range of learning problems using transfer.
  arXiv  Detail & Related papers  (2020-06-23T01:48:23Z)
• Exploring and Predicting Transferability across NLP Tasks [115.6278033699853]
  We study the transferability between 33 NLP tasks across three broad classes of problems. Our results show that transfer learning is more beneficial than previously thought. We also develop task embeddings that can be used to predict the most transferable source tasks for a given target task.
  arXiv  Detail & Related papers  (2020-05-02T09:39:36Z)
• Multi-Domain Spoken Language Understanding Using Domain- and Task-Aware Parameterization [78.93669377251396]
  Spoken language understanding has been addressed as a supervised learning problem, where a set of training data is available for each domain. One existing approach solves the problem by conducting multi-domain learning, using shared parameters for joint training across domains. We propose to improve the parameterization of this method by using domain-specific and task-specific model parameters.
  arXiv  Detail & Related papers  (2020-04-30T15:15:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.