Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering

• URL: http://arxiv.org/abs/2210.16495v1
• Date: Sat, 29 Oct 2022 05:11:45 GMT
• Title: Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering
• Authors: Deepanway Ghosal, Navonil Majumder, Rada Mihalcea, Soujanya Poria
• Abstract summary: We show that classifying (question, true answer) as positive instances and (question, false answer) as negative instances is significantly more effective across various models and datasets. Our DeBERTa binary classification model reaches the top or close to the top performance on public leaderboards for different tasks.
• Score: 43.35258958775454
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We propose a simple refactoring of multi-choice question answering (MCQA) tasks as a series of binary classifications. The MCQA task is generally performed by scoring each (question, answer) pair normalized over all the pairs, and then selecting the answer from the pair that yield the highest score. For n answer choices, this is equivalent to an n-class classification setup where only one class (true answer) is correct. We instead show that classifying (question, true answer) as positive instances and (question, false answer) as negative instances is significantly more effective across various models and datasets. We show the efficacy of our proposed approach in different tasks -- abductive reasoning, commonsense question answering, science question answering, and sentence completion. Our DeBERTa binary classification model reaches the top or close to the top performance on public leaderboards for these tasks. The source code of the proposed approach is available at https://github.com/declare-lab/TEAM.

Related papers

• Answering Ambiguous Questions via Iterative Prompting [84.3426020642704]
  In open-domain question answering, due to the ambiguity of questions, multiple plausible answers may exist. One approach is to directly predict all valid answers, but this can struggle with balancing relevance and diversity. We present AmbigPrompt to address the imperfections of existing approaches to answering ambiguous questions.
  arXiv  Detail & Related papers  (2023-07-08T04:32:17Z)
• BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answering [11.99004747630325]
  Unsupervised commonsense reasoning (UCR) is becoming increasingly popular as the construction of commonsense reasoning datasets is expensive. We propose to transform the downstream multiple choice question answering task into a simpler binary classification task by ranking all candidate answers according to their reasonableness.
  arXiv  Detail & Related papers  (2023-05-25T10:59:47Z)
• RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm [6.82469220191368]
  Answer selection (AS) is a critical subtask of the open-domain question answering (QA) problem. The present paper proposes a method called RLAS-BIABC for AS, which is established on attention mechanism-based long short-term memory (LSTM) and the bidirectional encoder representations from transformers (BERT) word embedding.
  arXiv  Detail & Related papers  (2023-01-07T08:33:05Z)
• Answer Generation for Retrieval-based Question Answering Systems [80.28727681633096]
  We train a sequence to sequence transformer model to generate an answer from a candidate set. Our tests on three English AS2 datasets show improvement up to 32 absolute points in accuracy over the state of the art.
  arXiv  Detail & Related papers  (2021-06-02T05:45:49Z)
• A Semantic-based Method for Unsupervised Commonsense Question Answering [40.18557352036813]
  Unsupervised commonsense question answering is appealing since it does not rely on any labeled task data. We present a novel SEmantic-based Question Answering method (SEQA) for unsupervised commonsense question answering.
  arXiv  Detail & Related papers  (2021-05-31T08:21:52Z)
• Generative Context Pair Selection for Multi-hop Question Answering [60.74354009152721]
  We propose a generative context selection model for multi-hop question answering. Our proposed generative passage selection model has a better performance (4.9% higher than baseline) on adversarial held-out set.
  arXiv  Detail & Related papers  (2021-04-18T07:00:48Z)
• Few-Shot Question Answering by Pretraining Span Selection [58.31911597824848]
  We explore the more realistic few-shot setting, where only a few hundred training examples are available. We show that standard span selection models perform poorly, highlighting the fact that current pretraining objective are far removed from question answering. Our findings indicate that careful design of pretraining schemes and model architecture can have a dramatic effect on performance in the few-shot settings.
  arXiv  Detail & Related papers  (2021-01-02T11:58:44Z)
• Generating Correct Answers for Progressive Matrices Intelligence Tests [88.78821060331582]
  Raven's Progressive Matrices are multiple-choice intelligence tests, where one tries to complete the missing location in a $3times 3$ grid of abstract images. Previous attempts to address this test have focused solely on selecting the right answer out of the multiple choices. In this work, we focus, instead, on generating a correct answer given the grid, without seeing the choices, which is a harder task, by definition.
  arXiv  Detail & Related papers  (2020-11-01T13:21:07Z)
• Classifier Combination Approach for Question Classification for Bengali Question Answering System [17.567099458403707]
  The work presented here demonstrates that the combination of multiple models achieve better classification performance than those obtained with existing individual models for the question classification task in Bengali. We have exploited state-of-the-art multiple model combination techniques, i.e., ensemble, stacking and voting, to increase QC accuracy. The approach presented here could be used in other Indo-Aryan or Indic languages to develop a question answering system.
  arXiv  Detail & Related papers  (2020-08-31T13:39:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.