Machine Reading Comprehension: The Role of Contextualized Language
Models and Beyond
- URL: http://arxiv.org/abs/2005.06249v1
- Date: Wed, 13 May 2020 10:58:50 GMT
- Title: Machine Reading Comprehension: The Role of Contextualized Language
Models and Beyond
- Authors: Zhuosheng Zhang, Hai Zhao, Rui Wang
- Abstract summary: Machine reading comprehension (MRC) aims to teach machines to read and comprehend human languages.
With the burst of deep neural networks and the evolution of contextualized language models (CLMs), the research of MRC has experienced two significant breakthroughs.
- Score: 85.53037880415734
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Machine reading comprehension (MRC) aims to teach machines to read and
comprehend human languages, which is a long-standing goal of natural language
processing (NLP). With the burst of deep neural networks and the evolution of
contextualized language models (CLMs), the research of MRC has experienced two
significant breakthroughs. MRC and CLM, as a phenomenon, have a great impact on
the NLP community. In this survey, we provide a comprehensive and comparative
review on MRC covering overall research topics about 1) the origin and
development of MRC and CLM, with a particular focus on the role of CLMs; 2) the
impact of MRC and CLM to the NLP community; 3) the definition, datasets, and
evaluation of MRC; 4) general MRC architecture and technical methods in the
view of two-stage Encoder-Decoder solving architecture from the insights of the
cognitive process of humans; 5) previous highlights, emerging topics, and our
empirical analysis, among which we especially focus on what works in different
periods of MRC researches. We propose a full-view categorization and new
taxonomies on these topics. The primary views we have arrived at are that 1)
MRC boosts the progress from language processing to understanding; 2) the rapid
improvement of MRC systems greatly benefits from the development of CLMs; 3)
the theme of MRC is gradually moving from shallow text matching to cognitive
reasoning.
Related papers
- Retrieval-Enhanced Machine Learning: Synthesis and Opportunities [60.34182805429511]
Retrieval-enhancement can be extended to a broader spectrum of machine learning (ML)
This work introduces a formal framework of this paradigm, Retrieval-Enhanced Machine Learning (REML), by synthesizing the literature in various domains in ML with consistent notations which is missing from the current literature.
The goal of this work is to equip researchers across various disciplines with a comprehensive, formally structured framework of retrieval-enhanced models, thereby fostering interdisciplinary future research.
arXiv Detail & Related papers (2024-07-17T20:01:21Z) - DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large
Language Models [85.4852517178828]
We present DualFocus, a framework for integrating macro and micro perspectives within multi-modal large language models (MLLMs)
We demonstrate DualFocus's superiority in balancing detailed examination with holistic insight, significantly reducing hallucination instances in MLLMs.
arXiv Detail & Related papers (2024-02-22T18:26:02Z) - Bridging Causal Discovery and Large Language Models: A Comprehensive
Survey of Integrative Approaches and Future Directions [10.226735765284852]
Causal discovery (CD) and Large Language Models (LLMs) represent two emerging fields of study with significant implications for artificial intelligence.
This paper presents a comprehensive survey of the integration of LLMs, such as GPT4, into CD tasks.
arXiv Detail & Related papers (2024-02-16T20:48:53Z) - Large Model Based Referring Camouflaged Object Detection [51.80619142347807]
Referring camouflaged object detection (Ref-COD) is a recently-proposed problem aiming to segment out specified camouflaged objects matched with a textual or visual reference.
Our motivation is to make full use of the semantic intelligence and intrinsic knowledge of recent Multimodal Large Language Models (MLLMs) to decompose this complex task in a human-like way.
We propose a large-model-based Multi-Level Knowledge-Guided multimodal method for Ref-COD termed MLKG.
arXiv Detail & Related papers (2023-11-28T13:45:09Z) - A Comprehensive Survey on Multi-hop Machine Reading Comprehension
Approaches [0.0]
Machine reading comprehension (MRC) is a long-standing topic in natural language processing (NLP)
Recently studies focus on multi-hop MRC which is a more challenging extension of MRC.
This study aims to investigate recent advances in the multi-hop MRC approaches based on 31 studies from 2018 to 2022.
arXiv Detail & Related papers (2022-12-08T04:51:54Z) - Understanding Attention in Machine Reading Comprehension [56.72165932439117]
This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final performance.
We perform quantitative analyses on SQuAD (English) and CMRC 2018 (Chinese), two span-extraction MRC datasets, on top of BERT, ALBERT, and ELECTRA.
We discover that em passage-to-question and em passage understanding attentions are the most important ones, showing strong correlations to the final performance.
arXiv Detail & Related papers (2021-08-26T04:23:57Z) - A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and
Benchmark Datasets [5.54205518616467]
Machine Reading (MRC) is a challenging Natural Language Processing(NLP) research field with wide real-world applications.
A lot of MRC models have already surpassed human performance on various benchmark datasets.
This shows the need for improving existing datasets, evaluation metrics, and models to move current MRC models toward "real" understanding.
arXiv Detail & Related papers (2020-06-21T19:18:54Z) - Enhancing Answer Boundary Detection for Multilingual Machine Reading
Comprehension [86.1617182312817]
We propose two auxiliary tasks in the fine-tuning stage to create additional phrase boundary supervision.
A mixed Machine Reading task, which translates the question or passage to other languages and builds cross-lingual question-passage pairs.
A language-agnostic knowledge masking task by leveraging knowledge phrases mined from web.
arXiv Detail & Related papers (2020-04-29T10:44:00Z) - A Study of the Tasks and Models in Machine Reading Comprehension [3.6985039575807246]
This report reviews some representative simple-reasoning and complex-reasoning MRC tasks.
It also proposes some open problems for the future research.
arXiv Detail & Related papers (2020-01-23T16:11:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.