SkillQG: Learning to Generate Question for Reading Comprehension Assessment

• URL: http://arxiv.org/abs/2305.04737v1
• Date: Mon, 8 May 2023 14:40:48 GMT
• Title: SkillQG: Learning to Generate Question for Reading Comprehension Assessment
• Authors: Xiaoqiang Wang, Bang Liu, Siliang Tang, Lingfei Wu
• Abstract summary: We present a question generation framework with controllable comprehension types for assessing and improving machine reading comprehension models. We first frame the comprehension type of questions based on a hierarchical skill-based schema, then formulate $textttSkillQG$ as a skill-conditioned question generator. Empirical results demonstrate that $textttSkillQG$ outperforms baselines in terms of quality, relevance, and skill-controllability.
• Score: 54.48031346496593
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present $\textbf{$\texttt{SkillQG}$}$: a question generation framework with controllable comprehension types for assessing and improving machine reading comprehension models. Existing question generation systems widely differentiate questions by $\textit{literal}$ information such as question words and answer types to generate semantically relevant questions for a given context. However, they rarely consider the $\textit{comprehension}$ nature of questions, i.e. the different comprehension capabilities embodied by different questions. In comparison, our $\texttt{SkillQG}$ is able to tailor a fine-grained assessment and improvement to the capabilities of question answering models built on it. Specifically, we first frame the comprehension type of questions based on a hierarchical skill-based schema, then formulate $\texttt{SkillQG}$ as a skill-conditioned question generator. Furthermore, to improve the controllability of generation, we augment the input text with question focus and skill-specific knowledge, which are constructed by iteratively prompting the pre-trained language models. Empirical results demonstrate that $\texttt{SkillQG}$ outperforms baselines in terms of quality, relevance, and skill-controllability while showing a promising performance boost in downstream question answering task.

Related papers

• Diversity Enhanced Narrative Question Generation for Storybooks [4.043005183192124]
  We introduce a multi-question generation model (mQG) capable of generating multiple, diverse, and answerable questions. To validate the answerability of the generated questions, we employ a SQuAD2.0 fine-tuned question answering model. mQG shows promising results across various evaluation metrics, among strong baselines.
  arXiv  Detail & Related papers  (2023-10-25T08:10:04Z)
• Improving Question Generation with Multi-level Content Planning [70.37285816596527]
  This paper addresses the problem of generating questions from a given context and an answer, specifically focusing on questions that require multi-hop reasoning across an extended context. We propose MultiFactor, a novel QG framework based on multi-level content planning. Specifically, MultiFactor includes two components: FA-model, which simultaneously selects key phrases and generates full answers, and Q-model which takes the generated full answer as an additional input to generate questions.
  arXiv  Detail & Related papers  (2023-10-20T13:57:01Z)
• Open-Set Knowledge-Based Visual Question Answering with Inference Paths [79.55742631375063]
  The purpose of Knowledge-Based Visual Question Answering (KB-VQA) is to provide a correct answer to the question with the aid of external knowledge bases. We propose a new retriever-ranker paradigm of KB-VQA, Graph pATH rankER (GATHER for brevity) Specifically, it contains graph constructing, pruning, and path-level ranking, which not only retrieves accurate answers but also provides inference paths that explain the reasoning process.
  arXiv  Detail & Related papers  (2023-10-12T09:12:50Z)
• ChatPRCS: A Personalized Support System for English Reading Comprehension based on ChatGPT [3.847982502219679]
  This paper presents a novel personalized support system for reading comprehension, referred to as ChatPRCS. ChatPRCS employs methods including reading comprehension proficiency prediction, question generation, and automatic evaluation.
  arXiv  Detail & Related papers  (2023-09-22T11:46:44Z)
• Discourse Analysis via Questions and Answers: Parsing Dependency Structures of Questions Under Discussion [57.43781399856913]
  This work adopts the linguistic framework of Questions Under Discussion (QUD) for discourse analysis. We characterize relationships between sentences as free-form questions, in contrast to exhaustive fine-grained questions. We develop the first-of-its-kind QUD that derives a dependency structure of questions over full documents.
  arXiv  Detail & Related papers  (2022-10-12T03:53:12Z)
• Automatic question generation based on sentence structure analysis using machine learning approach [0.0]
  This article introduces our framework for generating factual questions from unstructured text in the English language. It uses a combination of traditional linguistic approaches based on sentence patterns with several machine learning methods. The framework also includes a question evaluation module which estimates the quality of generated questions.
  arXiv  Detail & Related papers  (2022-05-25T14:35:29Z)
• QRelScore: Better Evaluating Generated Questions with Deeper Understanding of Context-aware Relevance [54.48031346496593]
  We propose $textbfQRelScore$, a context-aware evaluation metric for $underlinetextbfRel$evance evaluation metric. Based on off-the-shelf language models such as BERT and GPT2, QRelScore employs both word-level hierarchical matching and sentence-level prompt-based generation. Compared with existing metrics, our experiments demonstrate that QRelScore is able to achieve a higher correlation with human judgments while being much more robust to adversarial samples.
  arXiv  Detail & Related papers  (2022-04-29T07:39:53Z)
• Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask [3.470121495099]
  We study Question Generation (QG) for reading comprehension where inferential questions are critical. We propose a two-step model (HTA-WTA) that takes advantage of previous datasets. We show that the HTA-WTA model tests for strong SCRS by asking deep inferential questions.
  arXiv  Detail & Related papers  (2022-04-06T15:52:24Z)
• Inquisitive Question Generation for High Level Text Comprehension [60.21497846332531]
  We introduce INQUISITIVE, a dataset of 19K questions that are elicited while a person is reading through a document. We show that readers engage in a series of pragmatic strategies to seek information. We evaluate question generation models based on GPT-2 and show that our model is able to generate reasonable questions.
  arXiv  Detail & Related papers  (2020-10-04T19:03:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.