Generative Language Models for Paragraph-Level Question Generation

• URL: http://arxiv.org/abs/2210.03992v1
• Date: Sat, 8 Oct 2022 10:24:39 GMT
• Title: Generative Language Models for Paragraph-Level Question Generation
• Authors: Asahi Ushio and Fernando Alva-Manchego and Jose Camacho-Collados
• Abstract summary: Powerful generative models have led to recent progress in question generation (QG) It is difficult to measure advances in QG research since there are no standardized resources that allow a uniform comparison among approaches. We introduce QG-Bench, a benchmark for QG that unifies existing question answering datasets by converting them to a standard QG setting.
• Score: 79.31199020420827
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Powerful generative models have led to recent progress in question generation (QG). However, it is difficult to measure advances in QG research since there are no standardized resources that allow a uniform comparison among approaches. In this paper, we introduce QG-Bench, a multilingual and multidomain benchmark for QG that unifies existing question answering datasets by converting them to a standard QG setting. It includes general-purpose datasets such as SQuAD for English, datasets from ten domains and two styles, as well as datasets in eight different languages. Using QG-Bench as a reference, we perform an extensive analysis of the capabilities of language models for the task. First, we propose robust QG baselines based on fine-tuning generative language models. Then, we complement automatic evaluation based on standard metrics with an extensive manual evaluation, which in turn sheds light on the difficulty of evaluating QG models. Finally, we analyse both the domain adaptability of these models as well as the effectiveness of multilingual models in languages other than English. QG-Bench is released along with the fine-tuned models presented in the paper https://github.com/asahi417/lm-question-generation, which are also available as a demo https://autoqg.net/.

Related papers

• Cross-lingual Transfer for Automatic Question Generation by Learning Interrogative Structures in Target Languages [6.635572580071933]
  We propose a simple and efficient XLT-QG method that operates without the need for monolingual, parallel, or labeled data in the target language. Our method achieves performance comparable to GPT-3.5-turbo across different languages.
  arXiv  Detail & Related papers  (2024-10-04T07:29:35Z)
• A Practical Toolkit for Multilingual Question and Answer Generation [79.31199020420827]
  We introduce AutoQG, an online service for multilingual QAG, along with lmqg, an all-in-one Python package for model fine-tuning, generation, and evaluation. We also release QAG models in eight languages fine-tuned on a few variants of pre-trained encoder-decoder language models, which can be used online via AutoQG or locally via lmqg.
  arXiv  Detail & Related papers  (2023-05-27T08:42:37Z)
• An Empirical Comparison of LM-based Question and Answer Generation Methods [79.31199020420827]
  Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context. In this paper, we establish baselines with three different QAG methodologies that leverage sequence-to-sequence language model (LM) fine-tuning. Experiments show that an end-to-end QAG model, which is computationally light at both training and inference times, is generally robust and outperforms other more convoluted approaches.
  arXiv  Detail & Related papers  (2023-05-26T14:59:53Z)
• PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale [53.92008514395125]
  PAXQA (Projecting annotations for cross-lingual (x) QA) decomposes cross-lingual QA into two stages. We propose a novel use of lexically-constrained machine translation, in which constrained entities are extracted from the parallel bitexts. We show that models fine-tuned on these datasets outperform prior synthetic data generation models over several extractive QA datasets.
  arXiv  Detail & Related papers  (2023-04-24T15:46:26Z)
• Question Answering and Question Generation for Finnish [0.8426855646402236]
  We present the first neural QA and QG models that work with Finnish. To train the models, we automatically translate the SQuAD dataset. Using the synthetic data, together with the Finnish partition of the TyDi-QA dataset, we fine-tune several transformer-based models to both QA and QG.
  arXiv  Detail & Related papers  (2022-11-24T20:40:00Z)
• QAmeleon: Multilingual QA with Only 5 Examples [71.80611036543633]
  We show how to leverage pre-trained language models under a few-shot learning setting. Our approach, QAmeleon, uses a PLM to automatically generate multilingual data upon which QA models are trained. Prompt tuning the PLM for data synthesis with only five examples per language delivers accuracy superior to translation-based baselines.
  arXiv  Detail & Related papers  (2022-11-15T16:14:39Z)
• Cross-Lingual GenQA: A Language-Agnostic Generative Question Answering Approach for Open-Domain Question Answering [76.99585451345702]
  Open-Retrieval Generative Question Answering (GenQA) is proven to deliver high-quality, natural-sounding answers in English. We present the first generalization of the GenQA approach for the multilingual environment.
  arXiv  Detail & Related papers  (2021-10-14T04:36:29Z)
• An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers [3.4355075318742165]
  We show that multilingual, word-level QE models perform on par with the current language-specific models. In the cases of zero-shot and few-shot QE, we demonstrate that it is possible to accurately predict word-level quality for any given new language pair from models trained on other language pairs.
  arXiv  Detail & Related papers  (2021-05-31T23:21:10Z)
• Simplifying Paragraph-level Question Generation via Transformer Language Models [0.0]
  Question generation (QG) is a natural language generation task where a model is trained to ask questions corresponding to some input text. A single Transformer-based unidirectional language model leveraging transfer learning can be used to produce high quality questions. Our QG model, finetuned from GPT-2 Small, outperforms several paragraph-level QG baselines on the SQuAD dataset by 0.95 METEOR points.
  arXiv  Detail & Related papers  (2020-05-03T14:57:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.