UnifiedQA: Crossing Format Boundaries With a Single QA System

• URL: http://arxiv.org/abs/2005.00700v3
• Date: Wed, 7 Oct 2020 03:12:45 GMT
• Title: UnifiedQA: Crossing Format Boundaries With a Single QA System
• Authors: Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, Hannaneh Hajishirzi
• Abstract summary: We argue that such boundaries are artificial and perhaps unnecessary, given the reasoning abilities we seek to teach are not governed by the format. We build a single pre-trained QA model, UnifiedQA, that performs surprisingly well across 17 QA datasets spanning 4 diverse formats.
• Score: 84.63376743920003
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Question answering (QA) tasks have been posed using a variety of formats, such as extractive span selection, multiple choice, etc. This has led to format-specialized models, and even to an implicit division in the QA community. We argue that such boundaries are artificial and perhaps unnecessary, given the reasoning abilities we seek to teach are not governed by the format. As evidence, we use the latest advances in language modeling to build a single pre-trained QA model, UnifiedQA, that performs surprisingly well across 17 QA datasets spanning 4 diverse formats. UnifiedQA performs on par with 9 different models that were trained on individual datasets themselves. Even when faced with 12 unseen datasets of observed formats, UnifiedQA performs surprisingly well, showing strong generalization from its out-of-format training data. Finally, simply fine-tuning this pre-trained QA model into specialized models results in a new state of the art on 6 datasets, establishing UnifiedQA as a strong starting point for building QA systems.

Related papers

• Graph Guided Question Answer Generation for Procedural Question-Answering [29.169773816553153]
  We introduce a method for generating exhaustive and high-quality training data for task-specific question answering (QA) models. Key technological enabler is a novel mechanism for automatic question-answer generation from procedural text. We show that small models trained with our data achieve excellent performance on the target QA task, even exceeding that of GPT3 and ChatGPT.
  arXiv  Detail & Related papers  (2024-01-24T17:01:42Z)
• GSQA: An End-to-End Model for Generative Spoken Question Answering [54.418723701886115]
  We introduce the first end-to-end Generative Spoken Question Answering (GSQA) model that empowers the system to engage in abstractive reasoning. Our model surpasses the previous extractive model by 3% on extractive QA datasets. Our GSQA model shows the potential to generalize to a broad spectrum of questions, thus further expanding the spoken question answering capabilities of abstractive QA.
  arXiv  Detail & Related papers  (2023-12-15T13:33:18Z)
• Long-Tailed Question Answering in an Open World [46.67715607552547]
  We define Open Long-Tailed QA (OLTQA) as learning from long-tailed distributed data. We propose an OLTQA model that encourages knowledge sharing between head, tail and unseen tasks. On a large-scale OLTQA dataset, our model consistently outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2023-05-11T04:28:58Z)
• Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering [11.754328280233628]
  We argue that training explanation models independently of the QA model makes the explanations less grounded and limits performance. We propose a multitask learning approach towards a Unified Model for more grounded and consistent generation of both Answers and Explanations.
  arXiv  Detail & Related papers  (2023-01-25T19:29:19Z)
• QA4QG: Using Question Answering to Constrain Multi-Hop Question Generation [54.136509061542775]
  Multi-hop question generation (MQG) aims to generate complex questions which require reasoning over multiple pieces of information of the input passage. We propose a novel framework, QA4QG, a QA-augmented BART-based framework for MQG. Our results on the HotpotQA dataset show that QA4QG outperforms all state-of-the-art models.
  arXiv  Detail & Related papers  (2022-02-14T08:16:47Z)
• Unified Question Generation with Continual Lifelong Learning [41.81627903996791]
  Existing QG methods mainly focus on building or training models for specific QG datasets. We propose a model named UnifiedQG based on lifelong learning techniques, which can continually learn QG tasks. In addition, we transform the ability of a single trained Unified-QG model in improving $8$ Question Answering (QA) systems' performance.
  arXiv  Detail & Related papers  (2022-01-24T14:05:18Z)
• Learning to Perturb Word Embeddings for Out-of-distribution QA [55.103586220757464]
  We propose a simple yet effective DA method based on a noise generator, which learns to perturb the word embedding of the input questions and context without changing their semantics. We validate the performance of the QA models trained with our word embedding on a single source dataset, on five different target domains. Notably, the model trained with ours outperforms the model trained with more than 240K artificially generated QA pairs.
  arXiv  Detail & Related papers  (2021-05-06T14:12:26Z)
• Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs [62.71505254770827]
  We propose a conditional variational autoencoder (HCVAE) for generating QA pairs given unstructured texts as contexts. Our model obtains impressive performance gains over all baselines on both tasks, using only a fraction of data for training.
  arXiv  Detail & Related papers  (2020-05-28T08:26:06Z)
• Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering [98.48363619128108]
  We propose an unsupervised approach to training QA models with generated pseudo-training data. We show that generating questions for QA training by applying a simple template on a related, retrieved sentence rather than the original context sentence improves downstream QA performance.
  arXiv  Detail & Related papers  (2020-04-24T17:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.