Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning

• URL: http://arxiv.org/abs/2011.03292v2
• Date: Tue, 17 Nov 2020 06:26:33 GMT
• Title: Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning
• Authors: Yufan Jiang, Shuangzhi Wu, Jing Gong, Yahui Cheng, Peng Meng, Weiliang Lin, Zhibo Chen and Mu li
• Abstract summary: Multi-choice Machine Reading (MMRC) aims to select the correct answer from a set of options based on a given passage and question. It is non-trivial to transfer knowledge from other MRC tasks such as SQuAD, Dream. We reconstruct multi-choice to single-choice by training a binary classification to distinguish whether a certain answer is correct.
• Score: 18.81256990043713
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-choice Machine Reading Comprehension (MMRC) aims to select the correct answer from a set of options based on a given passage and question. Due to task specific of MMRC, it is non-trivial to transfer knowledge from other MRC tasks such as SQuAD, Dream. In this paper, we simply reconstruct multi-choice to single-choice by training a binary classification to distinguish whether a certain answer is correct. Then select the option with the highest confidence score. We construct our model upon ALBERT-xxlarge model and estimate it on the RACE dataset. During training, We adopt AutoML strategy to tune better parameters. Experimental results show that the single-choice is better than multi-choice. In addition, by transferring knowledge from other kinds of MRC tasks, our model achieves a new state-of-the-art results in both single and ensemble settings.

Related papers

• Differentiating Choices via Commonality for Multiple-Choice Question Answering [54.04315943420376]
  Multiple-choice question answering can provide valuable clues for choosing the right answer. Existing models often rank each choice separately, overlooking the context provided by other choices. We propose a novel model by differentiating choices through identifying and eliminating their commonality, called DCQA.
  arXiv  Detail & Related papers  (2024-08-21T12:05:21Z)
• Diversified Batch Selection for Training Acceleration [68.67164304377732]
  A prevalent research line, known as online batch selection, explores selecting informative subsets during the training process. vanilla reference-model-free methods involve independently scoring and selecting data in a sample-wise manner. We propose Diversified Batch Selection (DivBS), which is reference-model-free and can efficiently select diverse and representative samples.
  arXiv  Detail & Related papers  (2024-06-07T12:12:20Z)
• Transfer Learning Enhanced Single-choice Decision for Multi-choice Question Answering [27.601353412882258]
  Multi-choice Machine Reading (MMRC) aims to select the correct answer from a set of options based on a given passage and question. In this paper, we reconstruct multi-choice to single-choice by training a binary classification to distinguish whether a certain answer is correct. Our proposed method gets rid of the multi-choice framework and can leverage resources of other tasks.
  arXiv  Detail & Related papers  (2024-04-27T16:02:55Z)
• Large Language Models Are Not Robust Multiple Choice Selectors [117.72712117510953]
  Multiple choice questions (MCQs) serve as a common yet important task format in the evaluation of large language models (LLMs) This work shows that modern LLMs are vulnerable to option position changes due to their inherent "selection bias" We propose a label-free, inference-time debiasing method, called PriDe, which separates the model's prior bias for option IDs from the overall prediction distribution.
  arXiv  Detail & Related papers  (2023-09-07T17:44:56Z)
• Cost-Effective Online Contextual Model Selection [14.094350329970537]
  We formulate this task as an online contextual active model selection problem, where at each round the learner receives an unlabeled data point along with a context. The goal is to output the best model for any given context without obtaining an excessive amount of labels. We propose a contextual active model selection algorithm (CAMS), which relies on a novel uncertainty sampling query criterion defined on a given policy class for adaptive model selection.
  arXiv  Detail & Related papers  (2022-07-13T08:22:22Z)
• True Few-Shot Learning with Language Models [78.42578316883271]
  We evaluate the few-shot ability of LMs when held-out examples are unavailable. Our findings suggest that prior work significantly overestimated the true few-shot ability of LMs.
  arXiv  Detail & Related papers  (2021-05-24T17:55:51Z)
• Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question Answering Data [58.36305373100518]
  It is unclear whether subject-area question-answering data is useful for machine reading comprehension tasks. We collect a large-scale multi-subject multiple-choice question-answering dataset, ExamQA. We use incomplete and noisy snippets returned by a web search engine as the relevant context for each question-answering instance to convert it into a weakly-labeled MRC instance.
  arXiv  Detail & Related papers  (2021-02-01T23:18:58Z)
• DUMA: Reading Comprehension with Transposition Thinking [107.89721765056281]
  Multi-choice Machine Reading (MRC) requires model to decide the correct answer from a set of answer options when given a passage and a question. New DUal Multi-head Co-Attention (DUMA) model is inspired by human's transposition thinking process solving the multi-choice MRC problem.
  arXiv  Detail & Related papers  (2020-01-26T07:35:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.