Closed-book Question Generation via Contrastive Learning

• URL: http://arxiv.org/abs/2210.06781v1
• Date: Thu, 13 Oct 2022 06:45:46 GMT
• Title: Closed-book Question Generation via Contrastive Learning
• Authors: Xiangjue Dong, Jiaying Lu, Jianling Wang, James Caverlee
• Abstract summary: We propose a new QG model empowered by a contrastive learning module and an answer reconstruction module. We show how to leverage the proposed model to improve existing closed-book QA systems.
• Score: 20.644215991166895
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Question Generation (QG) is a fundamental NLP task for many downstream applications. Recent studies on open-book QG, where supportive question-context pairs are provided to models, have achieved promising progress. However, generating natural questions under a more practical closed-book setting that lacks these supporting documents still remains a challenge. In this work, to learn better representations from semantic information hidden in question-answer pairs under the closed-book setting, we propose a new QG model empowered by a contrastive learning module and an answer reconstruction module. We present a new closed-book QA dataset -- WikiCQA involving abstractive long answers collected from a wiki-style website. In the experiments, we validate the proposed QG model on both public datasets and the new WikiCQA dataset. Empirical results show that the proposed QG model outperforms baselines in both automatic evaluation and human evaluation. In addition, we show how to leverage the proposed model to improve existing closed-book QA systems. We observe that by pre-training a closed-book QA model on our generated synthetic QA pairs, significant QA improvement can be achieved on both seen and unseen datasets, which further demonstrates the effectiveness of our QG model for enhancing unsupervised and semi-supervised QA.

Related papers

• Graph Guided Question Answer Generation for Procedural Question-Answering [29.169773816553153]
  We introduce a method for generating exhaustive and high-quality training data for task-specific question answering (QA) models. Key technological enabler is a novel mechanism for automatic question-answer generation from procedural text. We show that small models trained with our data achieve excellent performance on the target QA task, even exceeding that of GPT3 and ChatGPT.
  arXiv  Detail & Related papers  (2024-01-24T17:01:42Z)
• QADYNAMICS: Training Dynamics-Driven Synthetic QA Diagnostic for Zero-Shot Commonsense Question Answering [48.25449258017601]
  State-of-the-art approaches fine-tune language models on QA pairs constructed from CommonSense Knowledge Bases. We propose QADYNAMICS, a training dynamics-driven framework for QA diagnostics and refinement.
  arXiv  Detail & Related papers  (2023-10-17T14:27:34Z)
• An Empirical Comparison of LM-based Question and Answer Generation Methods [79.31199020420827]
  Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context. In this paper, we establish baselines with three different QAG methodologies that leverage sequence-to-sequence language model (LM) fine-tuning. Experiments show that an end-to-end QAG model, which is computationally light at both training and inference times, is generally robust and outperforms other more convoluted approaches.
  arXiv  Detail & Related papers  (2023-05-26T14:59:53Z)
• Learning Answer Generation using Supervision from Automatic Question Answering Evaluators [98.9267570170737]
  We propose a novel training paradigm for GenQA using supervision from automatic QA evaluation models (GAVA) We evaluate our proposed methods on two academic and one industrial dataset, obtaining a significant improvement in answering accuracy over the previous state of the art.
  arXiv  Detail & Related papers  (2023-05-24T16:57:04Z)
• Improving Unsupervised Question Answering via Summarization-Informed Question Generation [47.96911338198302]
  Question Generation (QG) is the task of generating a plausible question for a passage, answer> pair. We make use of freely available news summary data, transforming declarative sentences into appropriate questions using dependency parsing, named entity recognition and semantic role labeling. The resulting questions are then combined with the original news articles to train an end-to-end neural QG model.
  arXiv  Detail & Related papers  (2021-09-16T13:08:43Z)
• Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study [45.9120218818558]
  We benchmark the research on the NarrativeQA dataset with experiments with cutting-edge ODQA techniques. This quantifies the challenges Book QA poses, as well as advances the published state-of-the-art with a $sim$7% absolute improvement on Rouge-L. Our findings indicate that the event-centric questions dominate this task, which exemplifies the inability of existing QA models to handle event-oriented scenarios.
  arXiv  Detail & Related papers  (2021-06-07T17:46:09Z)
• Harvesting and Refining Question-Answer Pairs for Unsupervised QA [95.9105154311491]
  We introduce two approaches to improve unsupervised Question Answering (QA) First, we harvest lexically and syntactically divergent questions from Wikipedia to automatically construct a corpus of question-answer pairs (named as RefQA) Second, we take advantage of the QA model to extract more appropriate answers, which iteratively refines data over RefQA.
  arXiv  Detail & Related papers  (2020-05-06T15:56:06Z)
• Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering [98.48363619128108]
  We propose an unsupervised approach to training QA models with generated pseudo-training data. We show that generating questions for QA training by applying a simple template on a related, retrieved sentence rather than the original context sentence improves downstream QA performance.
  arXiv  Detail & Related papers  (2020-04-24T17:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.