Mask and Cloze: Automatic Open Cloze Question Generation using a Masked Language Model

• URL: http://arxiv.org/abs/2205.07202v1
• Date: Sun, 15 May 2022 07:03:09 GMT
• Title: Mask and Cloze: Automatic Open Cloze Question Generation using a Masked Language Model
• Authors: Shoya Matsumori, Kohei Okuoka, Ryoichi Shibata, Minami Inoue, Yosuke Fukuchi, Michita Imai
• Abstract summary: In spite of its benefits, the open cloze test has been introduced only sporadically on the educational front. We developed CLOZER, an automatic open cloze question generator. A comparative experiment with human-generated questions also reveals that CLOZER can generate OCQs better than the average non-native English teacher.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open cloze questions have been attracting attention for both measuring the ability and facilitating the learning of L2 English learners. In spite of its benefits, the open cloze test has been introduced only sporadically on the educational front, largely because it is burdensome for teachers to manually create the questions. Unlike the more commonly used multiple choice questions (MCQ), open cloze questions are in free form and thus teachers have to ensure that only a ground truth answer and no additional words will be accepted in the blank. To help ease this burden, we developed CLOZER, an automatic open cloze question generator. In this work, we evaluate CLOZER through quantitative experiments on 1,600 answers and show statistically that it can successfully generate open cloze questions that only accept the ground truth answer. A comparative experiment with human-generated questions also reveals that CLOZER can generate OCQs better than the average non-native English teacher. Additionally, we conduct a field study at a local high school to clarify the benefits and hurdles when introducing CLOZER. The results demonstrate that while students found the application useful for their language learning. Finally, on the basis of our findings, we proposed several design improvements.

Related papers

• Does Multiple Choice Have a Future in the Age of Generative AI? A Posttest-only RCT [2.116573423199236]
  The role of multiple-choice questions (MCQs) as effective learning tools has been debated in past research. This study evaluates MCQs effectiveness relative to open-response questions, both individually and in combination, on learning. We find no significant learning differences across conditions at posttest, but tutors in the condition took significantly less time to complete instruction.
  arXiv  Detail & Related papers  (2024-12-13T16:37:20Z)
• Comparison of Large Language Models for Generating Contextually Relevant Questions [6.080820450677854]
  GPT-3.5, Llama 2-Chat 13B, and T5 XXL are compared in their ability to create questions from university slide text without fine-tuning. Results indicate that GPT-3.5 and Llama 2-Chat 13B outperform T5 XXL by a small margin, particularly in terms of clarity and question-answer alignment.
  arXiv  Detail & Related papers  (2024-07-30T06:23:59Z)
• Crafting Interpretable Embeddings by Asking LLMs Questions [89.49960984640363]
  Large language models (LLMs) have rapidly improved text embeddings for a growing array of natural-language processing tasks. We introduce question-answering embeddings (QA-Emb), embeddings where each feature represents an answer to a yes/no question asked to an LLM. We use QA-Emb to flexibly generate interpretable models for predicting fMRI voxel responses to language stimuli.
  arXiv  Detail & Related papers  (2024-05-26T22:30:29Z)
• Which questions should I answer? Salience Prediction of Inquisitive Questions [118.097974193544]
  We show that highly salient questions are empirically more likely to be answered in the same article. We further validate our findings by showing that answering salient questions is an indicator of summarization quality in news.
  arXiv  Detail & Related papers  (2024-04-16T21:33:05Z)
• Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations [70.6395572287422]
  Self-alignment method is capable of not only refusing to answer but also providing explanation to the unanswerability of unknown questions. We conduct disparity-driven self-curation to select qualified data for fine-tuning the LLM itself for aligning the responses to unknown questions as desired.
  arXiv  Detail & Related papers  (2024-02-23T02:24:36Z)
• FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation [92.43001160060376]
  We study the factuality of large language models (LLMs) in the context of answering questions that test current world knowledge. We introduce FreshQA, a novel dynamic QA benchmark encompassing a diverse range of question and answer types. We benchmark a diverse array of both closed and open-source LLMs under a two-mode evaluation procedure that allows us to measure both correctness and hallucination. Motivated by these results, we present FreshPrompt, a simple few-shot prompting method that substantially boosts the performance of an LLM on FreshQA.
  arXiv  Detail & Related papers  (2023-10-05T00:04:12Z)
• Covering Uncommon Ground: Gap-Focused Question Generation for Answer Assessment [75.59538732476346]
  We focus on the problem of generating such gap-focused questions (GFQs) automatically. We define the task, highlight key desired aspects of a good GFQ, and propose a model that satisfies these.
  arXiv  Detail & Related papers  (2023-07-06T22:21:42Z)
• Improving Reading Comprehension Question Generation with Data Augmentation and Overgenerate-and-rank [3.854023945160742]
  Automated answer-aware reading comprehension question generation has significant potential to scale up learner support in educational activities. One key technical challenge in this setting is that there can be multiple questions, sometimes very different from each other, with the same answer. We propose 1) a data augmentation method that enriches the training dataset with diverse questions given the same context and answer and 2) an overgenerate-and-rank method to select the best question from a pool of candidates.
  arXiv  Detail & Related papers  (2023-06-15T04:23:25Z)
• I Do Not Understand What I Cannot Define: Automatic Question Generation With Pedagogically-Driven Content Selection [0.08602553195689512]
  Posing questions about what learners have read is a well-established way of fostering their text comprehension. Many textbooks lack self-assessment questions because authoring them is time-consuming and expensive. This paper introduces a novel pedagogically meaningful content selection mechanism to find question-worthy sentences and answers in arbitrary textbook contents.
  arXiv  Detail & Related papers  (2021-10-08T13:29:13Z)
• Generating Answer Candidates for Quizzes and Answer-Aware Question Generators [16.44011627249311]
  We propose a model that can generate a specified number of answer candidates for a given passage of text. Our experiments show that our proposed answer candidate generation model outperforms several baselines.
  arXiv  Detail & Related papers  (2021-08-29T19:33:51Z)
• Differentiable Open-Ended Commonsense Reasoning [80.94997942571838]
  We study open-ended commonsense reasoning (OpenCSR) using as a resource only a corpus of commonsense facts written in natural language. As an approach to OpenCSR, we propose DrFact, an efficient Differentiable model for multi-hop Reasoning over knowledge Facts.
  arXiv  Detail & Related papers  (2020-10-24T10:07:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.