DeepQR: Neural-based Quality Ratings for Learnersourced Multiple-Choice Questions

• URL: http://arxiv.org/abs/2111.10058v1
• Date: Fri, 19 Nov 2021 05:58:39 GMT
• Title: DeepQR: Neural-based Quality Ratings for Learnersourced Multiple-Choice Questions
• Authors: Lin Ni, Qiming Bao, Xiaoxuan Li, Qianqian Qi, Paul Denny, Jim Warren, Michael Witbrock, Jiamou Liu
• Abstract summary: We propose a novel neural-network model for automated question quality rating (AQQR) DeepQR is trained using multiple-choice-question (MCQ) datasets collected from PeerWise, a widely-used learnersourcing platform. experiments on datasets collected from eight university-level courses illustrate that DeepQR has superior performance over six comparative models.
• Score: 6.506382411474777
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated question quality rating (AQQR) aims to evaluate question quality through computational means, thereby addressing emerging challenges in online learnersourced question repositories. Existing methods for AQQR rely solely on explicitly-defined criteria such as readability and word count, while not fully utilising the power of state-of-the-art deep-learning techniques. We propose DeepQR, a novel neural-network model for AQQR that is trained using multiple-choice-question (MCQ) datasets collected from PeerWise, a widely-used learnersourcing platform. Along with designing DeepQR, we investigate models based on explicitly-defined features, or semantic features, or both. We also introduce a self-attention mechanism to capture semantic correlations between MCQ components, and a contrastive-learning approach to acquire question representations using quality ratings. Extensive experiments on datasets collected from eight university-level courses illustrate that DeepQR has superior performance over six comparative models.

Related papers

• Multimodal Reranking for Knowledge-Intensive Visual Question Answering [77.24401833951096]
  We introduce a multi-modal reranker to improve the ranking quality of knowledge candidates for answer generation. Experiments on OK-VQA and A-OKVQA show that multi-modal reranker from distant supervision provides consistent improvements.
  arXiv  Detail & Related papers  (2024-07-17T02:58:52Z)
• QADYNAMICS: Training Dynamics-Driven Synthetic QA Diagnostic for Zero-Shot Commonsense Question Answering [48.25449258017601]
  State-of-the-art approaches fine-tune language models on QA pairs constructed from CommonSense Knowledge Bases. We propose QADYNAMICS, a training dynamics-driven framework for QA diagnostics and refinement.
  arXiv  Detail & Related papers  (2023-10-17T14:27:34Z)
• SQUARE: Automatic Question Answering Evaluation using Multiple Positive and Negative References [73.67707138779245]
  We propose a new evaluation metric: SQuArE (Sentence-level QUestion AnsweRing Evaluation) We evaluate SQuArE on both sentence-level extractive (Answer Selection) and generative (GenQA) QA systems.
  arXiv  Detail & Related papers  (2023-09-21T16:51:30Z)
• Toward Unsupervised Realistic Visual Question Answering [70.67698100148414]
  We study the problem of realistic VQA (RVQA), where a model has to reject unanswerable questions (UQs) and answer answerable ones (AQs) We first point out 2 drawbacks in current RVQA research, where (1) datasets contain too many unchallenging UQs and (2) a large number of annotated UQs are required for training. We propose a new testing dataset, RGQA, which combines AQs from an existing VQA dataset with around 29K human-annotated UQs. This combines pseudo UQs obtained by randomly pairing images and questions, with an
  arXiv  Detail & Related papers  (2023-03-09T06:58:29Z)
• A Comparative Study of Question Answering over Knowledge Bases [2.6135123648293717]
  Question answering over knowledge bases (KBQA) has become a popular approach to help users extract information from knowledge bases. We provide a comparative study of six representative KBQA systems on eight benchmark datasets. We propose an advanced mapping algorithm to aid existing models in achieving superior results.
  arXiv  Detail & Related papers  (2022-11-15T14:23:47Z)
• Modern Question Answering Datasets and Benchmarks: A Survey [5.026863544662493]
  Question Answering (QA) is one of the most important natural language processing (NLP) tasks. It aims using NLP technologies to generate a corresponding answer to a given question based on the massive unstructured corpus. In this paper, we investigate influential QA datasets that have been released in the era of deep learning.
  arXiv  Detail & Related papers  (2022-06-30T05:53:56Z)
• CONVIQT: Contrastive Video Quality Estimator [63.749184706461826]
  Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms. Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner. Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning.
  arXiv  Detail & Related papers  (2022-06-29T15:22:01Z)
• OpenQA: Hybrid QA System Relying on Structured Knowledge Base as well as Non-structured Data [15.585969737147892]
  We propose an intelligent question-answering system based on structured KB and unstructured data, called OpenQA. We integrate KBQA structured question answering based on semantic parsing and deep representation learning, and two-stage unstructured question answering based on retrieval and neural machine reading comprehension into OpenQA.
  arXiv  Detail & Related papers  (2021-12-31T09:15:39Z)
• Learning Transformer Features for Image Quality Assessment [53.51379676690971]
  We propose a unified IQA framework that utilizes CNN backbone and transformer encoder to extract features. The proposed framework is compatible with both FR and NR modes and allows for a joint training scheme.
  arXiv  Detail & Related papers  (2021-12-01T13:23:00Z)
• EQG-RACE: Examination-Type Question Generation [21.17100754955864]
  We propose an innovative Examination-type Question Generation approach (EQG-RACE) to generate exam-like questions based on a dataset extracted from RACE. Two main strategies are employed in EQG-RACE for dealing with discrete answer information and reasoning among long contexts. Experimental results show a state-of-the-art performance of EQG-RACE, which is apparently superior to the baselines.
  arXiv  Detail & Related papers  (2020-12-11T03:52:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.