A Feasibility Study of Answer-Unaware Question Generation for Education

• URL: http://arxiv.org/abs/2203.08685v1
• Date: Wed, 16 Mar 2022 15:16:18 GMT
• Title: A Feasibility Study of Answer-Unaware Question Generation for Education
• Abstract summary: We show that a significant portion of errors in such systems arise from asking irrelevant or uninterpretable questions. We find that giving these models human-written summaries instead of the original text results in a significant increase in acceptability of generated questions.
• Score: 14.434418277566383
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We conduct a feasibility study into the applicability of answer-unaware question generation models to textbook passages. We show that a significant portion of errors in such systems arise from asking irrelevant or uninterpretable questions and that such errors can be ameliorated by providing summarized input. We find that giving these models human-written summaries instead of the original text results in a significant increase in acceptability of generated questions (33% $\rightarrow$ 83%) as determined by expert annotators. We also find that, in the absence of human-written summaries, automatic summarization can serve as a good middle ground.

Related papers

• What is a Good Question? Utility Estimation with LLM-based Simulations [37.87879572754863]
  QUEST simulates a learning environment that enables the quantification of a question's utility. We find that questions generated by models trained with rejection sampling based on question utility result in exam scores that are higher by at least 20%.
  arXiv  Detail & Related papers  (2025-02-24T18:08:41Z)
• Localizing and Mitigating Errors in Long-form Question Answering [79.63372684264921]
  Long-form question answering (LFQA) aims to provide thorough and in-depth answers to complex questions, enhancing comprehension. This work introduces HaluQuestQA, the first hallucination dataset with localized error annotations for human-written and model-generated LFQA answers.
  arXiv  Detail & Related papers  (2024-07-16T17:23:16Z)
• The Structure of Financial Equity Research Reports -- Identification of the Most Frequently Asked Questions in Financial Analyst Reports to Automate Equity Research Using Llama 3 and GPT-4 [0.0]
  The study analyzes 72 ERRs sentence-by-sentence, classifying their 48.7% sentences into 169 unique question archetypes.<n>We did not predefine the questions but derived them solely from the statements in the ERRs.<n>The research confirms that the current writing process of ERRs can likely benefit from additional automation, improving quality and efficiency.
  arXiv  Detail & Related papers  (2024-07-04T15:58:02Z)
• Automated Question Generation for Science Tests in Arabic Language Using NLP Techniques [5.233512464561313]
  This research presents an innovative Arabic question-generation system built upon a three-stage process. The proposed approach and results show a precision of 83.50%, a recall of 78.68%, and an Fl score of 80.95%.
  arXiv  Detail & Related papers  (2024-06-11T20:27:45Z)
• Groundedness in Retrieval-augmented Long-form Generation: An Empirical Study [61.74571814707054]
  We evaluate whether every generated sentence is grounded in retrieved documents or the model's pre-training data. Across 3 datasets and 4 model families, our findings reveal that a significant fraction of generated sentences are consistently ungrounded. Our results show that while larger models tend to ground their outputs more effectively, a significant portion of correct answers remains compromised by hallucinations.
  arXiv  Detail & Related papers  (2024-04-10T14:50:10Z)
• ExpertQA: Expert-Curated Questions and Attributed Answers [51.68314045809179]
  We conduct human evaluation of responses from a few representative systems along various axes of attribution and factuality. We collect expert-curated questions from 484 participants across 32 fields of study, and then ask the same experts to evaluate generated responses to their own questions. The output of our analysis is ExpertQA, a high-quality long-form QA dataset with 2177 questions spanning 32 fields, along with verified answers and attributions for claims in the answers.
  arXiv  Detail & Related papers  (2023-09-14T16:54:34Z)
• Covering Uncommon Ground: Gap-Focused Question Generation for Answer Assessment [75.59538732476346]
  We focus on the problem of generating such gap-focused questions (GFQs) automatically. We define the task, highlight key desired aspects of a good GFQ, and propose a model that satisfies these.
  arXiv  Detail & Related papers  (2023-07-06T22:21:42Z)
• 'John ate 5 apples' != 'John ate some apples': Self-Supervised Paraphrase Quality Detection for Algebraic Word Problems [5.682665111938764]
  This paper introduces the novel task of scoring paraphrases for Algebraic Word Problems (AWP) We propose ParaQD, a self-supervised paraphrase quality detection method using novel data augmentations. Our method outperforms existing state-of-the-art self-supervised methods by up to 32% while also demonstrating impressive zero-shot performance.
  arXiv  Detail & Related papers  (2022-06-16T16:01:59Z)
• Few-shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems [22.167776818471026]
  We show that our personalized corrective feedback system has the potential to improve Generative Question Answering systems. Our model vastly outperforms both simple and strong baselines in terms of student learning gains by 45% and 23% respectively when tested in a real dialogue-based ITS.
  arXiv  Detail & Related papers  (2022-06-08T22:59:23Z)
• Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-Centric Summarization [67.1483219601714]
  We propose a novel question generation method that first learns the question type distribution of an input story paragraph. We finetune a pre-trained transformer-based sequence-to-sequence model using silver samples composed by educational question-answer pairs. Our work indicates the necessity of decomposing question type distribution learning and event-centric summary generation for educational question generation.
  arXiv  Detail & Related papers  (2022-03-27T02:21:19Z)
• Reinforcement Learning for Abstractive Question Summarization with Question-aware Semantic Rewards [20.342580435464072]
  We introduce a reinforcement learning-based framework for abstractive question summarization. We propose two novel rewards obtained from the downstream tasks of (i) question-type identification and (ii) question-focus recognition. These rewards ensure the generation of semantically valid questions and encourage the inclusion of key medical entities/foci in the question summary.
  arXiv  Detail & Related papers  (2021-07-01T02:06:46Z)
• A New Score for Adaptive Tests in Bayesian and Credal Networks [64.80185026979883]
  A test is adaptive when its sequence and number of questions is dynamically tuned on the basis of the estimated skills of the taker. We present an alternative family of scores, based on the mode of the posterior probabilities, and hence easier to explain.
  arXiv  Detail & Related papers  (2021-05-25T20:35:42Z)
• Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus [23.676748207014903]
  We propose Answer-Clue-Style-aware Question Generation (ACS-QG) It aims at automatically generating high-quality and diverse question-answer pairs from unlabeled text corpus at scale. We can generate 2.8 million quality-assured question-answer pairs from a million sentences found in Wikipedia.
  arXiv  Detail & Related papers  (2020-01-27T05:27:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.