Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond

• URL: http://arxiv.org/abs/2005.06249v1
• Date: Wed, 13 May 2020 10:58:50 GMT
• Title: Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• Authors: Zhuosheng Zhang, Hai Zhao, Rui Wang
• Abstract summary: Machine reading comprehension (MRC) aims to teach machines to read and comprehend human languages. With the burst of deep neural networks and the evolution of contextualized language models (CLMs), the research of MRC has experienced two significant breakthroughs.
• Score: 85.53037880415734
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine reading comprehension (MRC) aims to teach machines to read and comprehend human languages, which is a long-standing goal of natural language processing (NLP). With the burst of deep neural networks and the evolution of contextualized language models (CLMs), the research of MRC has experienced two significant breakthroughs. MRC and CLM, as a phenomenon, have a great impact on the NLP community. In this survey, we provide a comprehensive and comparative review on MRC covering overall research topics about 1) the origin and development of MRC and CLM, with a particular focus on the role of CLMs; 2) the impact of MRC and CLM to the NLP community; 3) the definition, datasets, and evaluation of MRC; 4) general MRC architecture and technical methods in the view of two-stage Encoder-Decoder solving architecture from the insights of the cognitive process of humans; 5) previous highlights, emerging topics, and our empirical analysis, among which we especially focus on what works in different periods of MRC researches. We propose a full-view categorization and new taxonomies on these topics. The primary views we have arrived at are that 1) MRC boosts the progress from language processing to understanding; 2) the rapid improvement of MRC systems greatly benefits from the development of CLMs; 3) the theme of MRC is gradually moving from shallow text matching to cognitive reasoning.

Related papers

• Improving Causal Reasoning in Large Language Models: A Survey [16.55801836321059]
  Causal reasoning is a crucial aspect of intelligence, essential for problem-solving, decision-making, and understanding the world. Large language models (LLMs) can generate rationales for their outputs, but their ability to reliably perform causal reasoning remains uncertain.
  arXiv  Detail & Related papers  (2024-10-22T04:18:19Z)
• Understanding the Role of LLMs in Multimodal Evaluation Benchmarks [77.59035801244278]
  This paper investigates the role of the Large Language Model (LLM) backbone in Multimodal Large Language Models (MLLMs) evaluation. Our study encompasses four diverse MLLM benchmarks and eight state-of-the-art MLLMs. Key findings reveal that some benchmarks allow high performance even without visual inputs and up to 50% of error rates can be attributed to insufficient world knowledge in the LLM backbone.
  arXiv  Detail & Related papers  (2024-10-16T07:49:13Z)
• From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Models [56.9134620424985]
  Cross-modal reasoning (CMR) is increasingly recognized as a crucial capability in the progression toward more sophisticated artificial intelligence systems. The recent trend of deploying Large Language Models (LLMs) to tackle CMR tasks has marked a new mainstream of approaches for enhancing their effectiveness. This survey offers a nuanced exposition of current methodologies applied in CMR using LLMs, classifying these into a detailed three-tiered taxonomy.
  arXiv  Detail & Related papers  (2024-09-19T02:51:54Z)
• Retrieval-Enhanced Machine Learning: Synthesis and Opportunities [60.34182805429511]
  Retrieval-enhancement can be extended to a broader spectrum of machine learning (ML) This work introduces a formal framework of this paradigm, Retrieval-Enhanced Machine Learning (REML), by synthesizing the literature in various domains in ML with consistent notations which is missing from the current literature. The goal of this work is to equip researchers across various disciplines with a comprehensive, formally structured framework of retrieval-enhanced models, thereby fostering interdisciplinary future research.
  arXiv  Detail & Related papers  (2024-07-17T20:01:21Z)
• Bridging Causal Discovery and Large Language Models: A Comprehensive Survey of Integrative Approaches and Future Directions [10.226735765284852]
  Causal discovery (CD) and Large Language Models (LLMs) represent two emerging fields of study with significant implications for artificial intelligence. This paper presents a comprehensive survey of the integration of LLMs, such as GPT4, into CD tasks.
  arXiv  Detail & Related papers  (2024-02-16T20:48:53Z)
• Discovery of the Hidden World with Large Language Models [95.58823685009727]
  This paper presents Causal representatiOn AssistanT (COAT) that introduces large language models (LLMs) to bridge the gap. LLMs are trained on massive observations of the world and have demonstrated great capability in extracting key information from unstructured data. COAT also adopts CDs to find causal relations among the identified variables as well as to provide feedback to LLMs to iteratively refine the proposed factors.
  arXiv  Detail & Related papers  (2024-02-06T12:18:54Z)
• A Comprehensive Survey on Multi-hop Machine Reading Comprehension Approaches [0.0]
  Machine reading comprehension (MRC) is a long-standing topic in natural language processing (NLP) Recently studies focus on multi-hop MRC which is a more challenging extension of MRC. This study aims to investigate recent advances in the multi-hop MRC approaches based on 31 studies from 2018 to 2022.
  arXiv  Detail & Related papers  (2022-12-08T04:51:54Z)
• Multilingual Multi-Aspect Explainability Analyses on Machine Reading Comprehension Models [76.48370548802464]
  This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final MRC system performance. We discover that passage-to-question and passage understanding attentions are the most important ones in the question answering process. Through comprehensive visualizations and case studies, we also observe several general findings on the attention maps, which can be helpful to understand how these models solve the questions.
  arXiv  Detail & Related papers  (2021-08-26T04:23:57Z)
• A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets [5.54205518616467]
  Machine Reading (MRC) is a challenging Natural Language Processing(NLP) research field with wide real-world applications. A lot of MRC models have already surpassed human performance on various benchmark datasets. This shows the need for improving existing datasets, evaluation metrics, and models to move current MRC models toward "real" understanding.
  arXiv  Detail & Related papers  (2020-06-21T19:18:54Z)
• A Study of the Tasks and Models in Machine Reading Comprehension [3.6985039575807246]
  This report reviews some representative simple-reasoning and complex-reasoning MRC tasks. It also proposes some open problems for the future research.
  arXiv  Detail & Related papers  (2020-01-23T16:11:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.