BERT-based distractor generation for Swedish reading comprehension questions using a small-scale dataset

• URL: http://arxiv.org/abs/2108.03973v1
• Date: Mon, 9 Aug 2021 12:15:47 GMT
• Title: BERT-based distractor generation for Swedish reading comprehension questions using a small-scale dataset
• Authors: Dmytro Kalpakchi and Johan Boye
• Abstract summary: We present a new BERT-based method for automatically generating distractors using only a small-scale dataset. Evaluation shows that from a student's perspective, our method generated one or more plausible distractors for more than 50% of the MCQs in our test set.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An important part when constructing multiple-choice questions (MCQs) for reading comprehension assessment are the distractors, the incorrect but preferably plausible answer options. In this paper, we present a new BERT-based method for automatically generating distractors using only a small-scale dataset. We also release a new such dataset of Swedish MCQs (used for training the model), and propose a methodology for assessing the generated distractors. Evaluation shows that from a student's perspective, our method generated one or more plausible distractors for more than 50% of the MCQs in our test set. From a teacher's perspective, about 50% of the generated distractors were deemed appropriate. We also do a thorough analysis of the results.

Related papers

• FarsiMCQGen: a Persian Multiple-choice Question Generation Framework [2.026379197206863]
  This paper introduces FarsiMCQGen, an innovative approach for generating Persian-language multiple-choice questions (MCQs)<n>Our methodology combines candidate generation, filtering, and ranking techniques to build a model that generates answer choices resembling those in real MCQs.<n>We leverage advanced methods, including Transformers and knowledge graphs, integrated with rule-based approaches to craft credible distractors that challenge test-takers.
  arXiv  Detail & Related papers  (2025-10-16T20:52:07Z)
• Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction [1.9949730506194254]
  In designing multiple-choice questions (MCQs) in education, creating plausible distractors is crucial for identifying students' misconceptions and gaps in knowledge. This study presents a pipeline for training a model to generate distractors that are more likely to be selected by students.
  arXiv  Detail & Related papers  (2025-01-21T10:20:39Z)
• DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions [42.148511874019256]
  We introduce DiVERT, a novel variational approach that learns an interpretable representation of errors behind distractors in math multiple-choice questions (MCQs) We show that DiVERT, despite using a base open-source LLM with 7B parameters, outperforms state-of-the-art approaches using GPT-4o on downstream distractor generation. We also conduct a human evaluation with math educators and find that DiVERT leads to error labels that are of comparable quality to human-authored ones.
  arXiv  Detail & Related papers  (2024-06-27T17:37:31Z)
• FairytaleQA Translated: Enabling Educational Question and Answer Generation in Less-Resourced Languages [0.0]
  This paper introduces machine-translated versions of FairytaleQA, a renowned QA dataset designed to assess and enhance narrative comprehension skills in young children. We employ fine-tuned, modest-scale models to establish benchmarks for both Question Generation (QG) and QA tasks within the translated datasets. We present a case study proposing a model for generating question-answer pairs, with an evaluation incorporating quality metrics such as question well-formedness, answerability, relevance, and children suitability.
  arXiv  Detail & Related papers  (2024-06-06T16:31:47Z)
• Improving Automated Distractor Generation for Math Multiple-choice Questions with Overgenerate-and-rank [44.04217284677347]
  We propose a novel method to enhance the quality of generated distractors through overgenerate-and-rank. Our ranking model increases alignment with human-authored distractors, although human-authored ones are still preferred over generated ones.
  arXiv  Detail & Related papers  (2024-04-19T00:25:44Z)
• TQ-Net: Mixed Contrastive Representation Learning For Heterogeneous Test Questions [18.186909839033017]
  Test questions (TQ) are usually heterogeneous and multi-modal, e.g., some of them may only contain text, while others half contain images with information beyond their literal description. In this paper, we first improve previous text-only representation with a two-stage unsupervised instance level contrastive based pre-training method. Then, TQ-Net was proposed to fuse the content of images to the representation of heterogeneous data.
  arXiv  Detail & Related papers  (2023-03-09T10:55:48Z)
• Ensemble Transfer Learning for Multilingual Coreference Resolution [60.409789753164944]
  A problem that frequently occurs when working with a non-English language is the scarcity of annotated training data. We design a simple but effective ensemble-based framework that combines various transfer learning techniques. We also propose a low-cost TL method that bootstraps coreference resolution models by utilizing Wikipedia anchor texts.
  arXiv  Detail & Related papers  (2023-01-22T18:22:55Z)
• Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future [63.99570204416711]
  We reimplement 18 methods for detecting potential annotation errors and evaluate them on 9 English datasets. We define a uniform evaluation setup including a new formalization of the annotation error detection task. We release our datasets and implementations in an easy-to-use and open source software package.
  arXiv  Detail & Related papers  (2022-06-05T22:31:45Z)
• Automatic Short Math Answer Grading via In-context Meta-learning [2.0263791972068628]
  We study the problem of automatic short answer grading for students' responses to math questions. We use MathBERT, a variant of the popular language model BERT adapted to mathematical content, as our base model. Second, we use an in-context learning approach that provides scoring examples as input to the language model.
  arXiv  Detail & Related papers  (2022-05-30T16:26:02Z)
• Learning Compositional Representation for Few-shot Visual Question Answering [93.4061107793983]
  Current methods of Visual Question Answering perform well on the answers with an amount of training data but have limited accuracy on the novel ones with few examples. We propose to extract the attributes from the answers with enough data, which are later composed to constrain the learning of the few-shot ones. Experimental results on the VQA v2.0 validation dataset demonstrate the effectiveness of our proposed attribute network.
  arXiv  Detail & Related papers  (2021-02-21T10:16:24Z)
• Overcoming Language Priors with Self-supervised Learning for Visual Question Answering [62.88124382512111]
  Most Visual Question Answering (VQA) models suffer from the language prior problem. We introduce a self-supervised learning framework to solve this problem. Our method can significantly outperform the state-of-the-art.
  arXiv  Detail & Related papers  (2020-12-17T12:30:12Z)
• Uncertainty-aware Self-training for Text Classification with Few Labels [54.13279574908808]
  We study self-training as one of the earliest semi-supervised learning approaches to reduce the annotation bottleneck. We propose an approach to improve self-training by incorporating uncertainty estimates of the underlying neural network. We show our methods leveraging only 20-30 labeled samples per class for each task for training and for validation can perform within 3% of fully supervised pre-trained language models.
  arXiv  Detail & Related papers  (2020-06-27T08:13:58Z)
• Improving Multi-Turn Response Selection Models with Complementary Last-Utterance Selection by Instance Weighting [84.9716460244444]
  We consider utilizing the underlying correlation in the data resource itself to derive different kinds of supervision signals. We conduct extensive experiments in two public datasets and obtain significant improvement in both datasets.
  arXiv  Detail & Related papers  (2020-02-18T06:29:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.