A Comprehensive Survey on Multi-hop Machine Reading Comprehension Datasets and Metrics

• URL: http://arxiv.org/abs/2212.04070v1
• Date: Thu, 8 Dec 2022 04:42:59 GMT
• Title: A Comprehensive Survey on Multi-hop Machine Reading Comprehension Datasets and Metrics
• Authors: Azade Mohammadi (1), Reza Ramezani (2) and Ahmad Baraani (3) ((1) Candidate student in University of Isfahan, (2) Assistant Professor in University of Isfahan, (3) Professor of Computer Engineering in University of Isfahan)
• Abstract summary: Multi-hop Machine reading comprehension is a challenging task with aim of answering a question based on disjoint pieces of information. The evaluation metrics and datasets are a vital part of multi-hop MRC because it is not possible to train and evaluate models without them. This study aims to present a comprehensive survey on recent advances in multi-hop MRC evaluation metrics and datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multi-hop Machine reading comprehension is a challenging task with aim of answering a question based on disjoint pieces of information across the different passages. The evaluation metrics and datasets are a vital part of multi-hop MRC because it is not possible to train and evaluate models without them, also, the proposed challenges by datasets often are an important motivation for improving the existing models. Due to increasing attention to this field, it is necessary and worth reviewing them in detail. This study aims to present a comprehensive survey on recent advances in multi-hop MRC evaluation metrics and datasets. In this regard, first, the multi-hop MRC problem definition will be presented, then the evaluation metrics based on their multi-hop aspect will be investigated. Also, 15 multi-hop datasets have been reviewed in detail from 2017 to 2022, and a comprehensive analysis has been prepared at the end. Finally, open issues in this field have been discussed.

Related papers

• Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework [51.44863255495668]
  Multimodal reasoning is a critical component in the pursuit of artificial intelligence systems that exhibit human-like intelligence. We present Multi-Modal Reasoning(COCO-MMR) dataset, a novel dataset that encompasses an extensive collection of open-ended questions. We propose innovative techniques, including multi-hop cross-modal attention and sentence-level contrastive learning, to enhance the image and text encoders.
  arXiv  Detail & Related papers  (2023-07-24T08:58:25Z)
• A Comprehensive Survey on Multi-hop Machine Reading Comprehension Approaches [0.0]
  Machine reading comprehension (MRC) is a long-standing topic in natural language processing (NLP) Recently studies focus on multi-hop MRC which is a more challenging extension of MRC. This study aims to investigate recent advances in the multi-hop MRC approaches based on 31 studies from 2018 to 2022.
  arXiv  Detail & Related papers  (2022-12-08T04:51:54Z)
• Multilingual Multi-Aspect Explainability Analyses on Machine Reading Comprehension Models [76.48370548802464]
  This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final MRC system performance. We discover that passage-to-question and passage understanding attentions are the most important ones in the question answering process. Through comprehensive visualizations and case studies, we also observe several general findings on the attention maps, which can be helpful to understand how these models solve the questions.
  arXiv  Detail & Related papers  (2021-08-26T04:23:57Z)
• Why Machine Reading Comprehension Models Learn Shortcuts? [56.629192589376046]
  We argue that larger proportion of shortcut questions in training data make models rely on shortcut tricks excessively. A thorough empirical analysis shows that MRC models tend to learn shortcut questions earlier than challenging questions.
  arXiv  Detail & Related papers  (2021-06-02T08:43:12Z)
• Multi-Perspective Abstractive Answer Summarization [76.10437565615138]
  Community Question Answering forums contain a rich resource of answers to a wide range of questions. The goal of multi-perspective answer summarization is to produce a summary that includes all perspectives of the answer. This work introduces a novel dataset creation method to automatically create multi-perspective, bullet-point abstractive summaries.
  arXiv  Detail & Related papers  (2021-04-17T13:15:29Z)
• Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question Answering Data [58.36305373100518]
  It is unclear whether subject-area question-answering data is useful for machine reading comprehension tasks. We collect a large-scale multi-subject multiple-choice question-answering dataset, ExamQA. We use incomplete and noisy snippets returned by a web search engine as the relevant context for each question-answering instance to convert it into a weakly-labeled MRC instance.
  arXiv  Detail & Related papers  (2021-02-01T23:18:58Z)
• The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvements [14.707930573950787]
  We present MuSe-CaR, a first of its kind multimodal dataset. The data is publicly available as it recently served as the testing bed for the 1st Multimodal Sentiment Analysis Challenge.
  arXiv  Detail & Related papers  (2021-01-15T10:40:37Z)
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps [31.472490306390977]
  A multi-hop question answering dataset aims to test reasoning and inference skills by requiring a model to read multiple paragraphs to answer a given question. Previous studies revealed that many examples in existing multi-hop datasets do not require multi-hop reasoning to answer a question. We present a new multi-hop QA dataset, called 2WikiMultiHopQA, which uses structured and unstructured data.
  arXiv  Detail & Related papers  (2020-11-02T15:42:40Z)
• A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets [5.54205518616467]
  Machine Reading (MRC) is a challenging Natural Language Processing(NLP) research field with wide real-world applications. A lot of MRC models have already surpassed human performance on various benchmark datasets. This shows the need for improving existing datasets, evaluation metrics, and models to move current MRC models toward "real" understanding.
  arXiv  Detail & Related papers  (2020-06-21T19:18:54Z)
• Retrospective Reader for Machine Reading Comprehension [90.6069071495214]
  Machine reading comprehension (MRC) is an AI challenge that requires machine to determine the correct answers to questions based on a given passage. When unanswerable questions are involved in the MRC task, an essential verification module called verifier is especially required in addition to the encoder. This paper devotes itself to exploring better verifier design for the MRC task with unanswerable questions.
  arXiv  Detail & Related papers  (2020-01-27T11:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.