Long-range and hierarchical language predictions in brains and algorithms

• URL: http://arxiv.org/abs/2111.14232v1
• Date: Sun, 28 Nov 2021 20:26:07 GMT
• Title: Long-range and hierarchical language predictions in brains and algorithms
• Authors: Charlotte Caucheteux, Alexandre Gramfort, Jean-Remi King
• Abstract summary: We show that while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. This study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.
• Score: 82.81964713263483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning has recently made remarkable progress in natural language processing. Yet, the resulting algorithms remain far from competing with the language abilities of the human brain. Predictive coding theory offers a potential explanation to this discrepancy: while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. To test this hypothesis, we analyze the fMRI brain signals of 304 subjects each listening to 70min of short stories. After confirming that the activations of deep language algorithms linearly map onto those of the brain, we show that enhancing these models with long-range forecast representations improves their brain-mapping. The results further reveal a hierarchy of predictions in the brain, whereby the fronto-parietal cortices forecast more abstract and more distant representations than the temporal cortices. Overall, this study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.

Related papers

• Language Reconstruction with Brain Predictive Coding from fMRI Data [28.217967547268216]
  Theory of predictive coding suggests that human brain naturally engages in continuously predicting future word representations. textscPredFT achieves current state-of-the-art decoding performance with a maximum BLEU-1 score of $27.8%$.
  arXiv  Detail & Related papers  (2024-05-19T16:06:02Z)
• Causal Graph in Language Model Rediscovers Cortical Hierarchy in Human Narrative Processing [0.0]
  Previous studies have demonstrated that the features of language models can be mapped to fMRI brain activity. This raises the question: is there a commonality between information processing in language models and the human brain? To estimate information flow patterns in a language model, we examined the causal relationships between different layers.
  arXiv  Detail & Related papers  (2023-11-17T10:09:12Z)
• Why can neural language models solve next-word prediction? A mathematical perspective [53.807657273043446]
  We study a class of formal languages that can be used to model real-world examples of English sentences. Our proof highlights the different roles of the embedding layer and the fully connected component within the neural language model.
  arXiv  Detail & Related papers  (2023-06-20T10:41:23Z)
• Deep Learning Models to Study Sentence Comprehension in the Human Brain [0.1503974529275767]
  Recent artificial neural networks that process natural language achieve unprecedented performance in tasks requiring sentence-level understanding. We review works that compare these artificial language models with human brain activity and we assess the extent to which this approach has improved our understanding of the neural processes involved in natural language comprehension.
  arXiv  Detail & Related papers  (2023-01-16T10:31:25Z)
• Toward a realistic model of speech processing in the brain with self-supervised learning [67.7130239674153]
  Self-supervised algorithms trained on the raw waveform constitute a promising candidate. We show that Wav2Vec 2.0 learns brain-like representations with as little as 600 hours of unlabelled speech.
  arXiv  Detail & Related papers  (2022-06-03T17:01:46Z)
• Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects [82.81964713263483]
  A popular approach to decompose the neural bases of language consists in correlating, across individuals, the brain responses to different stimuli. Here, we show that a model-based approach can reach equivalent results within subjects exposed to natural stimuli.
  arXiv  Detail & Related papers  (2021-10-12T15:30:21Z)
• Voxel-level Importance Maps for Interpretable Brain Age Estimation [70.5330922395729]
  We focus on the task of brain age regression from 3D brain Magnetic Resonance (MR) images using a Convolutional Neural Network, termed prediction model. We implement a noise model which aims to add as much noise as possible to the input without harming the performance of the prediction model. We test our method on 13,750 3D brain MR images from the UK Biobank, and our findings are consistent with the existing neuropathology literature.
  arXiv  Detail & Related papers  (2021-08-11T18:08:09Z)
• Does injecting linguistic structure into language models lead to better alignment with brain recordings? [13.880819301385854]
  We evaluate whether language models align better with brain recordings if their attention is biased by annotations from syntactic or semantic formalisms. Our proposed approach enables the evaluation of more targeted hypotheses about the composition of meaning in the brain.
  arXiv  Detail & Related papers  (2021-01-29T14:42:02Z)
• Towards a Neural Model for Serial Order in Frontal Cortex: a Brain Theory from Memory Development to Higher-Level Cognition [53.816853325427424]
  We propose that the immature prefrontal cortex (PFC) use its primary functionality of detecting hierarchical patterns in temporal signals. Our hypothesis is that the PFC detects the hierarchical structure in temporal sequences in the form of ordinal patterns and use them to index information hierarchically in different parts of the brain. By doing so, it gives the tools to the language-ready brain for manipulating abstract knowledge and planning temporally ordered information.
  arXiv  Detail & Related papers  (2020-05-22T14:29:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.