Automatic Generation of Multiple-Choice Questions

• URL: http://arxiv.org/abs/2303.14576v1
• Date: Sat, 25 Mar 2023 22:45:54 GMT
• Title: Automatic Generation of Multiple-Choice Questions
• Authors: Cheng Zhang
• Abstract summary: We present two methods to tackle the challenge of QAP generations. A deep-learning-based end-to-end question generation system based on T5 Transformer with Preprocessing and Postprocessing Pipelines. A sequence-learning-based scheme to generate adequate QAPs via meta-sequence representations of sentences.
• Score: 7.310488568715925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Creating multiple-choice questions to assess reading comprehension of a given article involves generating question-answer pairs (QAPs) and adequate distractors. We present two methods to tackle the challenge of QAP generations: (1) A deep-learning-based end-to-end question generation system based on T5 Transformer with Preprocessing and Postprocessing Pipelines (TP3). We use the finetuned T5 model for our downstream task of question generation and improve accuracy using a combination of various NLP tools and algorithms in preprocessing and postprocessing to select appropriate answers and filter undesirable questions. (2) A sequence-learning-based scheme to generate adequate QAPs via meta-sequence representations of sentences. A meta-sequence is a sequence of vectors comprising semantic and syntactic tags. we devise a scheme called MetaQA to learn meta sequences from training data to form pairs of a meta sequence for a declarative sentence and a corresponding interrogative sentence. The TP3 works well on unseen data, which is complemented by MetaQA. Both methods can generate well-formed and grammatically correct questions. Moreover, we present a novel approach to automatically generate adequate distractors for a given QAP. The method is a combination of part-of-speech tagging, named-entity tagging, semantic-role labeling, regular expressions, domain knowledge bases, word embeddings, word edit distance, WordNet, and other algorithms.

Related papers

• Constructing Cloze Questions Generatively [2.2719421441459406]
  We present a generative method for constructing cloze questions from an article using neural networks and WordNet. CQG selects an answer key for a given sentence, segments it into a sequence of instances, generates instance-level distractor candidates (IDCs) using a transformer and sibling synsets. It then removes inappropriate IDCs, ranks the remaining IDCs based on contextual embedding similarities, as well as synset and lexical relatedness, forms distractor candidates by replacing instances with the corresponding top-ranked IDCs, and checks if they are legitimate phrases.
  arXiv  Detail & Related papers  (2024-10-05T18:55:38Z)
• Automated Generation of Multiple-Choice Cloze Questions for Assessing English Vocabulary Using GPT-turbo 3.5 [5.525336037820985]
  We evaluate a new method for automatically generating multiple-choice questions using large language models (LLM) The VocaTT engine is written in Python and comprises three basic steps: pre-processing target word lists, generating sentences and candidate word options, and finally selecting suitable word options. Results showed a 75% rate of well-formedness for sentences and 66.85% rate for suitable word options.
  arXiv  Detail & Related papers  (2024-03-04T14:24:47Z)
• Learning to Filter Context for Retrieval-Augmented Generation [75.18946584853316]
  Generation models are required to generate outputs given partially or entirely irrelevant passages. FILCO identifies useful context based on lexical and information-theoretic approaches. It trains context filtering models that can filter retrieved contexts at test time.
  arXiv  Detail & Related papers  (2023-11-14T18:41:54Z)
• Improving Question Generation with Multi-level Content Planning [70.37285816596527]
  This paper addresses the problem of generating questions from a given context and an answer, specifically focusing on questions that require multi-hop reasoning across an extended context. We propose MultiFactor, a novel QG framework based on multi-level content planning. Specifically, MultiFactor includes two components: FA-model, which simultaneously selects key phrases and generates full answers, and Q-model which takes the generated full answer as an additional input to generate questions.
  arXiv  Detail & Related papers  (2023-10-20T13:57:01Z)
• HPE:Answering Complex Questions over Text by Hybrid Question Parsing and Execution [92.69684305578957]
  We propose a framework of question parsing and execution on textual QA. The proposed framework can be viewed as a top-down question parsing followed by a bottom-up answer backtracking. Our experiments on MuSiQue, 2WikiQA, HotpotQA, and NQ show that the proposed parsing and hybrid execution framework outperforms existing approaches in supervised, few-shot, and zero-shot settings.
  arXiv  Detail & Related papers  (2023-05-12T22:37:06Z)
• Tag-Set-Sequence Learning for Generating Question-Answer Pairs [10.48660454637293]
  We present a new method called tag-set sequence learning to tackle the problem of generating silly questions for texts. We construct a system called TSS-Learner to learn tag-set sequences from given declarative sentences and the corresponding interrogative sentences. We show that TSS-Learner can indeed generate adequate QAPs for certain texts that transformer-based models do poorly.
  arXiv  Detail & Related papers  (2022-10-20T21:51:00Z)
• Paragraph-based Transformer Pre-training for Multi-Sentence Inference [99.59693674455582]
  We show that popular pre-trained transformers perform poorly when used for fine-tuning on multi-candidate inference tasks. We then propose a new pre-training objective that models the paragraph-level semantics across multiple input sentences.
  arXiv  Detail & Related papers  (2022-05-02T21:41:14Z)
• Discovering Non-monotonic Autoregressive Orderings with Variational Inference [67.27561153666211]
  We develop an unsupervised parallelizable learner that discovers high-quality generation orders purely from training data. We implement the encoder as a Transformer with non-causal attention that outputs permutations in one forward pass. Empirical results in language modeling tasks demonstrate that our method is context-aware and discovers orderings that are competitive with or even better than fixed orders.
  arXiv  Detail & Related papers  (2021-10-27T16:08:09Z)
• Generating Adequate Distractors for Multiple-Choice Questions [7.966913971277812]
  Our method is a combination of part-of-speech tagging, named-entity tagging, semantic-role labeling, regular expressions, domain knowledge bases, word embeddings, word edit distance, WordNet, and other algorithms. We show that, via experiments and by human judges, each MCQ has at least one adequate distractor and 84% of evaluations have three adequate distractors.
  arXiv  Detail & Related papers  (2020-10-23T20:47:58Z)
• Meta Sequence Learning for Generating Adequate Question-Answer Pairs [10.48660454637293]
  We present a learning scheme to generate adequate QAPs via meta-sequence representations of sentences. On a given declarative sentence, a trained MetaQA model converts it to a meta sequence, finds a matched MD, and uses the corresponding MIs and the input sentence to generate QAPs. We show that MetaQA generates efficiently over the official SAT practice reading tests a large number of syntactically and semantically correct QAPs with over 97% accuracy.
  arXiv  Detail & Related papers  (2020-10-04T16:28:13Z)
• Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching [87.97265483696613]
  We propose a new approach to sequence pair matching with Transformer, by learning head-wise matching representations on multiple levels. Experiments show that our proposed approach can achieve new state-of-the-art performance on multiple tasks.
  arXiv  Detail & Related papers  (2020-01-20T20:02:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.