Understanding Human Reading Comprehension with Brain Signals
- URL: http://arxiv.org/abs/2108.01360v2
- Date: Wed, 4 Aug 2021 01:07:13 GMT
- Title: Understanding Human Reading Comprehension with Brain Signals
- Authors: Ziyi Ye, Xiaohui Xie, Yiqun Liu, Zhihong Wang, Xuesong Chen, Min
Zhang, Shaoping Ma
- Abstract summary: Little is known about what happens in human brain during reading comprehension.
With the advances in brain imaging techniques such as EEG, it is possible to collect high-precision brain signals in almost real time.
- Score: 37.850947109491685
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reading comprehension is a complex cognitive process involving many human
brain activities. Plenty of works have studied the reading patterns and
attention allocation mechanisms in the reading process. However, little is
known about what happens in human brain during reading comprehension and how we
can utilize this information as implicit feedback to facilitate information
acquisition performance. With the advances in brain imaging techniques such as
EEG, it is possible to collect high-precision brain signals in almost real
time. With neuroimaging techniques, we carefully design a lab-based user study
to investigate brain activities during reading comprehension. Our findings show
that neural responses vary with different types of contents, i.e., contents
that can satisfy users' information needs and contents that cannot. We suggest
that various cognitive activities, e.g., cognitive loading, semantic-thematic
understanding, and inferential processing, at the micro-time scale during
reading comprehension underpin these neural responses. Inspired by these
detectable differences in cognitive activities, we construct supervised
learning models based on EEG features for two reading comprehension tasks:
answer sentence classification and answer extraction. Results show that it is
feasible to improve their performance with brain signals. These findings imply
that brain signals are valuable feedback for enhancing human-computer
interactions during reading comprehension.
Related papers
- Brain-like Functional Organization within Large Language Models [58.93629121400745]
The human brain has long inspired the pursuit of artificial intelligence (AI)
Recent neuroimaging studies provide compelling evidence of alignment between the computational representation of artificial neural networks (ANNs) and the neural responses of the human brain to stimuli.
In this study, we bridge this gap by directly coupling sub-groups of artificial neurons with functional brain networks (FBNs)
This framework links the AN sub-groups to FBNs, enabling the delineation of brain-like functional organization within large language models (LLMs)
arXiv Detail & Related papers (2024-10-25T13:15:17Z) - Decoding Linguistic Representations of Human Brain [21.090956290947275]
We present a taxonomy of brain-to-language decoding of both textual and speech formats.
This work integrates two types of research: neuroscience focusing on language understanding and deep learning-based brain decoding.
arXiv Detail & Related papers (2024-07-30T07:55:44Z) - MindSemantix: Deciphering Brain Visual Experiences with a Brain-Language Model [45.18716166499859]
Deciphering the human visual experience through brain activities captured by fMRI represents a compelling and cutting-edge challenge.
We introduce MindSemantix, a novel multi-modal framework that enables LLMs to comprehend visually-evoked semantic content in brain activity.
MindSemantix generates high-quality captions that are deeply rooted in the visual and semantic information derived from brain activity.
arXiv Detail & Related papers (2024-05-29T06:55:03Z) - BRACTIVE: A Brain Activation Approach to Human Visual Brain Learning [11.517021103782229]
We introduce Brain Activation Network (BRACTIVE), a transformer-based approach to studying the human visual brain.
The main objective of BRACTIVE is to align the visual features of subjects with corresponding brain representations via fMRI signals.
Our experiments demonstrate that BRACTIVE effectively identifies person-specific regions of interest, such as face and body-selective areas.
arXiv Detail & Related papers (2024-05-29T06:50:13Z) - Towards Decoding Brain Activity During Passive Listening of Speech [0.0]
We attempt to decode heard speech from intracranial electroencephalographic (iEEG) data using deep learning methods.
This approach diverges from the conventional focus on speech production and instead chooses to investigate neural representations of perceived speech.
Despite the approach not having achieved a breakthrough yet, the research sheds light on the potential of decoding neural activity during speech perception.
arXiv Detail & Related papers (2024-02-26T20:04:01Z) - Brain-Inspired Machine Intelligence: A Survey of
Neurobiologically-Plausible Credit Assignment [65.268245109828]
We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology.
We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors.
The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
arXiv Detail & Related papers (2023-12-01T05:20:57Z) - Toward a realistic model of speech processing in the brain with
self-supervised learning [67.7130239674153]
Self-supervised algorithms trained on the raw waveform constitute a promising candidate.
We show that Wav2Vec 2.0 learns brain-like representations with as little as 600 hours of unlabelled speech.
arXiv Detail & Related papers (2022-06-03T17:01:46Z) - CogAlign: Learning to Align Textual Neural Representations to Cognitive
Language Processing Signals [60.921888445317705]
We propose a CogAlign approach to integrate cognitive language processing signals into natural language processing models.
We show that CogAlign achieves significant improvements with multiple cognitive features over state-of-the-art models on public datasets.
arXiv Detail & Related papers (2021-06-10T07:10:25Z) - Attention Patterns Detection using Brain Computer Interfaces [1.174402845822043]
We propose a method to assess and quantify human attention levels and their effects on learning.
We employ a brain computer interface (BCI) capable of detecting brain wave activity and displaying the corresponding electroencephalograms (EEG)
We train recurrent neural networks (RNNS) to identify the type of activity an individual is performing.
arXiv Detail & Related papers (2020-05-20T11:55:37Z) - A Developmental Neuro-Robotics Approach for Boosting the Recognition of
Handwritten Digits [91.3755431537592]
Recent evidence shows that a simulation of the children's embodied strategies can improve the machine intelligence too.
This article explores the application of embodied strategies to convolutional neural network models in the context of developmental neuro-robotics.
arXiv Detail & Related papers (2020-03-23T14:55:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.